{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> https://www.kaggle.com/startupsci/titanic-data-science-solutions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Titanic问题的数据挖掘步骤\n",
    "## 首先介绍 Data Science Solutions书中处理的步骤\n",
    "\n",
    "\n",
    "1. 问题定义\n",
    "2. 构建训练集和测试集数据\n",
    "3. 准备和清洗数据\n",
    "4. 分析数据，识别特征模式\n",
    "5. 建模，预测并解决问题\n",
    "6. 可视化报告最终的方案\n",
    "7. 提交结果\n",
    "\n",
    "\n",
    "- 步骤可以打乱穿插"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. 问题定义\n",
    "\n",
    "来自Kaggle竞赛Titanic的介绍：\n",
    "> 已知训练集和测试集，训练集中是891个人的个人信息和是否存活，测试集中是个人信息，要求使用训练集训练我们的模型，判断测试集中的人是否存活\n",
    "\n",
    "一些需要知道的背景知识：\n",
    "- 1912年4月15日，Titanic首次航行撞冰山沉没2224个乘客中1505个丧生，存活率为32%\n",
    "- 没有足够的救生船是死亡率这么高的一个原因\n",
    "- 即使有一些幸运的成分，但是有些群体更容易存活，比如说女人、小孩和上层阶层"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. 解决流程\n",
    "- **Classifying**：对样本进行归类，不同类之间的相关性进行分析\n",
    "- **Correlating**：可以通过训练集现有的特征来解决问题。在这个数据集下，哪个特征对结果有贡献？从统计角度看特征和结果有什么相关性？改变特征值结果会不会变？这些可以在给定的数据集中计算。我们也可能需要确定结果和后续目标之间的相关性。特征相关性可以帮助构造特征，缺失值补全和纠错。\n",
    "- **Converting**：在建模阶段，需要准备数据。和模型算法有关，有的需要把所有特征的值都转化为数值型的。因此需要把文本数据或者序号数据都转换成数值性。\n",
    "- **Completing**：缺失值补全。\n",
    "- **Correcting**：分析给定数据集中错误或者不准确的值，并改正或者直接删除异常样本。一个方法是检测样本或者特征的异常。如果一个特征对结果没有贡献，可以删除这个特征。\n",
    "- **Creating**：根据一个或者一组特征构造新特征，依据是相关性、变换或者使目标完整。\n",
    "- **Charting**：根据数据和解决目标来选择图表。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 导包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# data analysis and wrangling\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import random as rnd\n",
    "\n",
    "# visualization\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# machine learning\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.svm import SVC, LinearSVC\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.linear_model import Perceptron\n",
    "from sklearn.linear_model import SGDClassifier\n",
    "from sklearn.tree import DecisionTreeClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 导入数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_df = pd.read_csv('../input/train.csv')\n",
    "test_df = pd.read_csv('../input/test.csv')\n",
    "combine = [train_df, test_df]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 通过数据描述来分析\n",
    "## 数据集中有哪些特征？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['PassengerId' 'Survived' 'Pclass' 'Name' 'Sex' 'Age' 'SibSp' 'Parch'\n",
      " 'Ticket' 'Fare' 'Cabin' 'Embarked']\n"
     ]
    }
   ],
   "source": [
    "print(train_df.columns.values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 类别特征\n",
    "把样本分成几个相似样本的集合。这些特征的值可以是名字、序数、百分比、按间隔划分。这个可以帮助我们选择合适的图进行可视化。\n",
    "- Survived, Sex, and Embarked. Ordinal: Pclass\n",
    "\n",
    "## 数值特征\n",
    "每个样本的值基本不同，非连续的、连续的、基于时间序列的。\n",
    "- Age, Fare. Discrete: SibSp, Parch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# preview the data\n",
    "train_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 混合类型的特征\n",
    "在同一个特征中存在数值型和数字字母型值。一般都是需要纠错的。\n",
    "- Ticket是数值型和数字字母型的混合， Cabin是数字字母型的\n",
    "\n",
    "## 包含错误的特征\n",
    "大数据集很难来预览，小数据可以方便的预览一些样本来看哪些是需要纠错的。\n",
    "- Name特征可能有错笔误，所以可以提取头衔、圆括号和引号作为 替代，或者短名来纠错"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>886</th>\n",
       "      <td>887</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Montvila, Rev. Juozas</td>\n",
       "      <td>male</td>\n",
       "      <td>27.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>211536</td>\n",
       "      <td>13.00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887</th>\n",
       "      <td>888</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Graham, Miss. Margaret Edith</td>\n",
       "      <td>female</td>\n",
       "      <td>19.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>112053</td>\n",
       "      <td>30.00</td>\n",
       "      <td>B42</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>888</th>\n",
       "      <td>889</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Johnston, Miss. Catherine Helen \"Carrie\"</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>W./C. 6607</td>\n",
       "      <td>23.45</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>890</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Behr, Mr. Karl Howell</td>\n",
       "      <td>male</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>111369</td>\n",
       "      <td>30.00</td>\n",
       "      <td>C148</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>890</th>\n",
       "      <td>891</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Dooley, Mr. Patrick</td>\n",
       "      <td>male</td>\n",
       "      <td>32.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>370376</td>\n",
       "      <td>7.75</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived  Pclass                                      Name  \\\n",
       "886          887         0       2                     Montvila, Rev. Juozas   \n",
       "887          888         1       1              Graham, Miss. Margaret Edith   \n",
       "888          889         0       3  Johnston, Miss. Catherine Helen \"Carrie\"   \n",
       "889          890         1       1                     Behr, Mr. Karl Howell   \n",
       "890          891         0       3                       Dooley, Mr. Patrick   \n",
       "\n",
       "        Sex   Age  SibSp  Parch      Ticket   Fare Cabin Embarked  \n",
       "886    male  27.0      0      0      211536  13.00   NaN        S  \n",
       "887  female  19.0      0      0      112053  30.00   B42        S  \n",
       "888  female   NaN      1      2  W./C. 6607  23.45   NaN        S  \n",
       "889    male  26.0      0      0      111369  30.00  C148        C  \n",
       "890    male  32.0      0      0      370376   7.75   NaN        Q  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 特征的缺失处理\n",
    "- Cabin > Age > Embarked ，训练集中缺失的数量排序\n",
    "- Cabin > Age 测试集中的缺失\n",
    "\n",
    "## 不同特征的数据类型\n",
    "可以帮助我们转换数据\n",
    "- 7个特征是整型或者浮点型，测试集中有6个\n",
    "- 5个特征是字符串型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 12 columns):\n",
      "PassengerId    891 non-null int64\n",
      "Survived       891 non-null int64\n",
      "Pclass         891 non-null int64\n",
      "Name           891 non-null object\n",
      "Sex            891 non-null object\n",
      "Age            714 non-null float64\n",
      "SibSp          891 non-null int64\n",
      "Parch          891 non-null int64\n",
      "Ticket         891 non-null object\n",
      "Fare           891 non-null float64\n",
      "Cabin          204 non-null object\n",
      "Embarked       889 non-null object\n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 83.6+ KB\n",
      "________________________________________\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 418 entries, 0 to 417\n",
      "Data columns (total 11 columns):\n",
      "PassengerId    418 non-null int64\n",
      "Pclass         418 non-null int64\n",
      "Name           418 non-null object\n",
      "Sex            418 non-null object\n",
      "Age            332 non-null float64\n",
      "SibSp          418 non-null int64\n",
      "Parch          418 non-null int64\n",
      "Ticket         418 non-null object\n",
      "Fare           417 non-null float64\n",
      "Cabin          91 non-null object\n",
      "Embarked       418 non-null object\n",
      "dtypes: float64(2), int64(4), object(5)\n",
      "memory usage: 36.0+ KB\n"
     ]
    }
   ],
   "source": [
    "train_df.info()\n",
    "print('_'*40)\n",
    "test_df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 样本中连续样本的分布\n",
    "这有助于我们确定除了其他早期见解之外，实际问题领域的培训数据集的代表性如何。\n",
    "\n",
    "- 总共891个样本，占Titanic样本(2224)中的41%\n",
    "- Survived 是一个0-1的类别特征\n",
    "- 样本中存活率为38%，实际是32%\n",
    "- 大多数乘客（75%）没有父母和小孩在船上\n",
    "- 大约30%的乘客有兄弟或者配偶在船上\n",
    "- 只有小于1%的乘客的船费大于512美元\n",
    "- 35-80岁的乘客小于1%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>714.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.383838</td>\n",
       "      <td>2.308642</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>0.523008</td>\n",
       "      <td>0.381594</td>\n",
       "      <td>32.204208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>257.353842</td>\n",
       "      <td>0.486592</td>\n",
       "      <td>0.836071</td>\n",
       "      <td>14.526497</td>\n",
       "      <td>1.102743</td>\n",
       "      <td>0.806057</td>\n",
       "      <td>49.693429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.420000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>223.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>20.125000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>7.910400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.454200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>668.500000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>31.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>512.329200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       PassengerId    Survived      Pclass         Age       SibSp  \\\n",
       "count   891.000000  891.000000  891.000000  714.000000  891.000000   \n",
       "mean    446.000000    0.383838    2.308642   29.699118    0.523008   \n",
       "std     257.353842    0.486592    0.836071   14.526497    1.102743   \n",
       "min       1.000000    0.000000    1.000000    0.420000    0.000000   \n",
       "25%     223.500000    0.000000    2.000000   20.125000    0.000000   \n",
       "50%     446.000000    0.000000    3.000000   28.000000    0.000000   \n",
       "75%     668.500000    1.000000    3.000000   38.000000    1.000000   \n",
       "max     891.000000    1.000000    3.000000   80.000000    8.000000   \n",
       "\n",
       "            Parch        Fare  \n",
       "count  891.000000  891.000000  \n",
       "mean     0.381594   32.204208  \n",
       "std      0.806057   49.693429  \n",
       "min      0.000000    0.000000  \n",
       "25%      0.000000    7.910400  \n",
       "50%      0.000000   14.454200  \n",
       "75%      0.000000   31.000000  \n",
       "max      6.000000  512.329200  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.describe()\n",
    "# Review survived rate using `percentiles=[.61, .62]` knowing our problem description mentions 38% survival rate.\n",
    "# Review Parch distribution using `percentiles=[.75, .8]`\n",
    "# SibSp distribution `[.68, .69]`\n",
    "# Age and Fare `[.1, .2, .3, .4, .5, .6, .7, .8, .9, .99]`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 类别特征的分布\n",
    "\n",
    "- Name在数据集中的每一个值都是唯一的(count=unique=891)\n",
    "- Sex变量有两个可能的值，male占65%(top=male, freq=577/count=891)\n",
    "- Cabin有一些样本的值是一样的，可能是有一些乘客共用一个船舱\n",
    "- Embarked有3个值，大多数乘客都是S(top=S)\n",
    "- Ticket有22%的重复率(unique=681)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>891</td>\n",
       "      <td>891</td>\n",
       "      <td>891</td>\n",
       "      <td>204</td>\n",
       "      <td>889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique</th>\n",
       "      <td>891</td>\n",
       "      <td>2</td>\n",
       "      <td>681</td>\n",
       "      <td>147</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>Graham, Mr. George Edward</td>\n",
       "      <td>male</td>\n",
       "      <td>CA. 2343</td>\n",
       "      <td>C23 C25 C27</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>freq</th>\n",
       "      <td>1</td>\n",
       "      <td>577</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>644</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             Name   Sex    Ticket        Cabin Embarked\n",
       "count                         891   891       891          204      889\n",
       "unique                        891     2       681          147        3\n",
       "top     Graham, Mr. George Edward  male  CA. 2343  C23 C25 C27        S\n",
       "freq                            1   577         7            4      644"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.describe(include=['O'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 根据数据分析的假设\n",
    "基于前面的数据分析做一些假设，可能会在后面进行验证\n",
    "\n",
    "- Correlating\n",
    "  1. 我们需要知道各个特征和结果的相关关系。先在工程中做一下，然后在模型中计算一下。\n",
    "- Completing\n",
    "  1. 根据和存活率的相关性来补全Age缺失值\n",
    "  2. 根据和存活率或者其他重要特征的相关性来补全Embarked缺失值\n",
    "- Correcting\n",
    "  1. Ticket要删掉，因为重复率22%，而且和存活特征相关性小\n",
    "  2. Cabin要删掉，因为训练集和测试集的缺失值太多\n",
    "  3. PassengerId要删掉，因为对存活结果没有贡献\n",
    "  4. Name特征不怎么标准，和存活没有直接关联，要删除\n",
    "- Creating\n",
    "  1. 用船上配偶和子女的数量（Parch和SibSp）的和构造家人数量（Family）特征\n",
    "  2. 在Name特征中提取头衔Title作为一个特征\n",
    "  3. 把Age特征按区间划分，从连续的数值型转化为类别特征\n",
    "  4. 把Fare按区间划分帮助分析\n",
    "- Classifying\n",
    "基于问题的描述增加一些假设\n",
    "  1. Women(Sex=female)更容易存活\n",
    "  2. Children (Age<?)更容易存活\n",
    "  3. The upper-class passengers (Pclass=1)更容易存活"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 转化特征来进行分析\n",
    "为了验证我们的观察和假设，通过转化特征来进行一个快速的特征分析。只能对不缺失的特征进行分析。只对categorical类别 (Sex), ordinal序数 (Pclass) or discrete不连续 (SibSp, Parch)的特征进行操作才有意义。\n",
    "\n",
    "1. Pclass=1和Survived有很高的相关性(>0.5)，将这个特征加入到我们的模型中\n",
    "2. Sex=female有很高的存活率74%\n",
    "3. SibSp和Parch有几个值是零相关，最好设置一个区间"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Parch</th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0.600000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.550847</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0.343658</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>0.200000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Parch  Survived\n",
       "3      3  0.600000\n",
       "1      1  0.550847\n",
       "2      2  0.500000\n",
       "0      0  0.343658\n",
       "5      5  0.200000\n",
       "4      4  0.000000\n",
       "6      6  0.000000"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df[['Pclass', 'Survived']].groupby(['Pclass'], as_index=False).mean().sort_values(by='Survived', ascending=False)\n",
    "train_df[[\"Sex\", \"Survived\"]].groupby(['Sex'], as_index=False).mean().sort_values(by='Survived', ascending=False)\n",
    "train_df[[\"SibSp\", \"Survived\"]].groupby(['SibSp'], as_index=False).mean().sort_values(by='Survived', ascending=False)\n",
    "train_df[[\"Parch\", \"Survived\"]].groupby(['Parch'], as_index=False).mean().sort_values(by='Survived', ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 可视化来分析\n",
    "\n",
    "### 分析数值型特征的相关性\n",
    "分析数值型特征和存活率目标之间的相关性\n",
    "用直方图来分析Age区间，用来验证小孩是不是更容易存活。\n",
    "\n",
    "### Observations\n",
    "1. 小孩(Age <=4)存活率高\n",
    "2. 老人(Age = 80)存活\n",
    "3. 很多15-25岁的人都没有存活\n",
    "4. 大多数乘客的年龄在15-35岁之间\n",
    "\n",
    "### Decisions\n",
    "上面的观察分析可以验证如下的假设并决策\n",
    "1. 把Age加入到模型中\n",
    "2. 补全Age的缺失值\n",
    "3. 划分Age的区间"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x6c2c850>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEWlJREFUeJzt3X+s3XV9x/HnS8qPDRxQvOk6wJUNAkEdIB2CqNsAt6pE\nyKwMwkxNujRLcMOp0TL/mGZugWRRyRxmjTg75rSIMhpcROTHpsahRRH5oYIIUga0IKCYRS2+98f5\nVq/0tvfc3nN6Puee5yM5Od+f57zvt/fd9/l8zud+vqkqJElqzXNGHYAkSTOxQEmSmmSBkiQ1yQIl\nSWqSBUqS1CQLlCSpSRaoIUvyziR3Jrk9yW1JXjKg131tkrUDeq2nB/Aa+ybZkOTeJLckWTb/yDQp\nJihPXpHkq0m2JVk5iLgWskWjDmAhS3IKcCbw4qr6cZLnAfvM4fxFVbVtpn1VtRHYOJhIB2I18ERV\nHZnkXOAS4E9GHJPGwITlyfeANwJvG3EcY8EW1HAtBR6rqh8DVNVjVfW/AEnu7xKRJMuT3NwtvyvJ\nFUm+CFyR5H+SvGD7Cya5uTv+jUk+kOTAJA8keU63f/8kDybZO8lvJ/lMkluTfD7JMd0xRyT5UpJv\nJHnPgH7Ws4D13fJVwOlJMqDX1sI2MXlSVfdX1e3AzwbxegudBWq4PgscnuTbSS5L8nt9nncscEZV\nnQdsAM4BSLIUWFpVm7YfWFVPAbcB21/7TOC6qvopsA74i6o6kd4ntsu6Yy4FPlhVLwIe3lkQXbLe\nNsPjjBkOPxR4sItpG/AUcEifP68m2yTliebALr4hqqqnk5wIvBz4A2BDkrVV9ZFZTt1YVf/XLV9J\nL4H/hl4CXjXD8RvodafdBJwLXJbkAOClwCemNWT27Z5PBV7XLV9BrztupvhfPkuc0ryZJ9oZC9SQ\nVdUzwM3AzUm+AawCPgJs4xct2P2eddqPpp3/UJLHk/wOveT68xneZiPw90kWAycCNwL7A09W1fE7\nC2222JN8HnjuDLveVlWfe9a2h4DDgc1JFgEHAo/P9h4STFSeaA7s4huiJEcnOWrapuOBB7rl++kl\nCfziU9rObADeDhzY9V//kqp6GvgKvS6Ja6vqmar6AfDdJK/vYkmS47pTvkjvEyTA+Tt706p6eVUd\nP8NjpqTbSO8/FYCVwI3lTMTqw4TliebAAjVcBwDrk9yV5HZ6febv6va9G7g0ySbgmVle5yp6iXLl\nLo7ZAPxp97zd+cDqJF8H7qQ3kAHgQuCC7pPqof3/OLt0OXBIknuBtwADGdqriTAxeZLkd5NsBl4P\n/HOSOwfxugtV/JArSWqRLShJUpMsUJKkJlmgJElNskBJkpq0RwvUihUrit7fFfjwsVAf82ae+JiA\nR1/2aIF67LHH9uTbSWPJPJF67OKTJDXJAiVJapIFSpLUJAuUJKlJFihJUpMsUJKkJnk/qAFatvbT\nO913/8Wv2YORSNL4swUlSWqSBUqS1CQLlCSpSRYoSVKTHCSxh+xqAAU4iEKSns0WlCSpSRYoSVKT\nLFCSpCZZoCRJTbJASZKaZIGSJDWpr2HmSe4Hfgg8A2yrquVJFgMbgGXA/cA5VfXEcMLcMxwKLknt\nmEsL6g+q6viqWt6trwVuqKqjgBu6dUmSBmI+XXxnAeu75fXA2fMPR5Kknn4LVAGfTXJrkjXdtiVV\n9XC3/AiwZKYTk6xJsinJpq1bt84zXGlhMk+kHfVboF5WVS8GXgVckOQV03dWVdErYjuoqnVVtbyq\nlk9NTc0vWmmBMk+kHfVVoKrqoe55C3A1cBLwaJKlAN3zlmEFKUmaPLMWqCT7J3nu9mXgD4E7gI3A\nqu6wVcA1wwpSkjR5+hlmvgS4Osn24/+9qj6T5CvAlUlWAw8A5wwvTEnSpJm1QFXVfcBxM2x/HDh9\nGEG1ara/k5IkDY4zSUiSmmSBkiQ1yQIlSWqSBUqS1CQLlCSpSRYoSVKTLFCSpCZZoCRJTbJASZKa\nZIGSJDXJAiVJapIFSpLUJAuUJKlJFihJUpMsUJKkJlmgJElN6rtAJdkrydeSXNutH5HkliT3JtmQ\nZJ/hhSlJmjRzaUFdCNw9bf0S4H1VdSTwBLB6kIFJkiZbXwUqyWHAa4APdesBTgOu6g5ZD5w9jAAl\nSZOp3xbU+4G3Az/r1g8Bnqyqbd36ZuDQmU5MsibJpiSbtm7dOq9gpYXKPJF2NGuBSnImsKWqbt2d\nN6iqdVW1vKqWT01N7c5LSAueeSLtaFEfx5wKvDbJq4H9gF8DLgUOSrKoa0UdBjw0vDAlSZNm1hZU\nVV1UVYdV1TLgXODGqjofuAlY2R22CrhmaFFKkibOfP4O6h3AW5LcS+87qcsHE5IkSf118f1cVd0M\n3Nwt3wecNPiQJElyJglJUqMsUJKkJlmgJElNskBJkpo0p0ESkjQXy9Z+epf777/4NXsoEo0jW1CS\npCZZoCRJTbKLT1LTZusmnI3diOPLFpQkqUm2oMbErj5F+glR0kJkC0qS1CQLlCSpSRYoSVKTLFCS\npCZZoCRJTbJASZKaNGuBSrJfki8n+XqSO5O8u9t+RJJbktybZEOSfYYfriRpUvTTgvoxcFpVHQcc\nD6xIcjJwCfC+qjoSeAJYPbwwJUmTZtYCVT1Pd6t7d48CTgOu6ravB84eSoSSpInU13dQSfZKchuw\nBbge+A7wZFVt6w7ZDBw6nBAlSZOor6mOquoZ4PgkBwFXA8f0+wZJ1gBrAJ7//OfvTowTYb4TYmq8\nTWqe+HuvXZnTKL6qehK4CTgFOCjJ9gJ3GPDQTs5ZV1XLq2r51NTUvIKVFirzRNpRP6P4prqWE0l+\nBXglcDe9QrWyO2wVcM2wgpQkTZ5+uviWAuuT7EWvoF1ZVdcmuQv4eJL3AF8DLh9inJKkCTNrgaqq\n24ETZth+H3DSMIKSJMn7QS0As33R7P2iJI0jpzqSJDXJFpQ0hlpoNTtEXMNmC0qS1CQLlCSpSRYo\nSVKTLFCSpCZZoCRJTbJASZKaZIGSJDXJAiVJapIFSpLUJGeSUBOzEkjSs9mCkiQ1yQIlSWqSBUqS\n1CQLlCSpSbMWqCSHJ7kpyV1J7kxyYbd9cZLrk9zTPR88/HAlSZOinxbUNuCtVXUscDJwQZJjgbXA\nDVV1FHBDty5J0kDMWqCq6uGq+mq3/EPgbuBQ4CxgfXfYeuDsYQUpSZo8c/oOKsky4ATgFmBJVT3c\n7XoEWLKTc9Yk2ZRk09atW+cRqrRwmSfSjvouUEkOAD4JvLmqfjB9X1UVUDOdV1Xrqmp5VS2fmpqa\nV7DSQmWeSDvqq0Al2ZtecfpoVX2q2/xokqXd/qXAluGEKEmaRP2M4gtwOXB3Vb132q6NwKpueRVw\nzeDDkyRNqn7m4jsVeAPwjSS3ddv+GrgYuDLJauAB4JzhhChJmkSzFqiq+gKQnew+fbDhSJLU40wS\nkqQmWaAkSU3yflATYLb7PUkLWT+//97zrE22oCRJTbJASZKaZIGSJDXJAiVJapKDJDSrXX3J7JfL\nC5eDazRqtqAkSU2yBSVJAzBbi9PehrmzBSVJapIFSpLUpOa6+GwmS5LAFpQkqVHNtaAkaU9zSH2b\nbEFJkprUzy3fP5xkS5I7pm1bnOT6JPd0zwcPN0xJ0qTpp4vvI8AHgH+dtm0tcENVXZxkbbf+jsGH\nN3cOspCkhWHWFlRV/Tfw/WdtPgtY3y2vB84ecFySpAm3u99BLamqh7vlR4AlOzswyZokm5Js2rp1\n626+nbSwmSfSjuY9SKKqCqhd7F9XVcuravnU1NR8305akMwTaUe7W6AeTbIUoHveMriQJEna/b+D\n2gisAi7unq8ZWESS5q2fv+txwJBa188w848BXwKOTrI5yWp6hemVSe4BzujWJUkamFlbUFV13k52\nnT7gWLQAOexf0u5yJglJUpMsUJKkJjlZrOZlvpNs7up8u/+0kNjdPXe2oCRJTbJASZKaZBefmmWX\niDTZbEFJkpo0di2oYX4pL0mj4uwfO7IFJUlqkgVKktSksevik/rlIItds7t7/Eza77QtKElSkyxQ\nkqQmWaAkSU2yQEmSmuQgCY0tv+SXFjZbUJKkJs2rBZVkBXApsBfwoary1u+StIDtyRkvdrsFlWQv\n4J+AVwHHAuclOXYgUUmSJt58uvhOAu6tqvuq6ifAx4GzBhOWJGnSpap278RkJbCiqv6sW38D8JKq\netOzjlsDrOlWjwa+tZOXfB7w2G4Fs+eNS6zGOVj9xPlYVa2Y6wvPIU/6jaMFxjlY4xInzB5rX3ky\n9FF8VbUOWDfbcUk2VdXyYcczCOMSq3EO1jDj7DdPhh3HIBnnYI1LnDC4WOfTxfcQcPi09cO6bZIk\nzdt8CtRXgKOSHJFkH+BcYONgwpIkTbrd7uKrqm1J3gRcR2+Y+Yer6s55xNJX90YjxiVW4xysVuJs\nJY7ZGOdgjUucMKBYd3uQhCRJw+RMEpKkJlmgJElNaqJAJVmR5FtJ7k2ydtTxbJfk8CQ3JbkryZ1J\nLuy2L05yfZJ7uueDRx0r9Gb3SPK1JNd260ckuaW7rhu6wSyjjvGgJFcl+WaSu5Oc0vD1/Kvu3/2O\nJB9Lst8or2mreQLjlSvjkCcwPrkyzDwZeYFqfMqkbcBbq+pY4GTggi62tcANVXUUcEO33oILgbun\nrV8CvK+qjgSeAFaPJKpfdinwmao6BjiOXrzNXc8khwJ/CSyvqhfSGwh0LiO6po3nCYxXroxDnsAY\n5MrQ86SqRvoATgGum7Z+EXDRqOPaSazXAK+k91f+S7ttS4FvNRDbYfR+YU8DrgVC7y+5F810nUcU\n44HAd+kG50zb3uL1PBR4EFhMb7TrtcAfjeqajlOedPE1mSvjkCddHGORK8POk5G3oPjFD7jd5m5b\nU5IsA04AbgGWVNXD3a5HgCUjCmu69wNvB37WrR8CPFlV27r1Fq7rEcBW4F+6LpYPJdmfBq9nVT0E\n/APwPeBh4CngVkZ3TcciT6D5XBmHPIExyZVh50kLBap5SQ4APgm8uap+MH1f9T4ijHSsfpIzgS1V\ndeso4+jDIuDFwAer6gTgRzyri6KF6wnQ9e2fRe8/it8A9gfmPMfepGk5V8YoT2BMcmXYedJCgWp6\nyqQke9NLuI9W1ae6zY8mWdrtXwpsGVV8nVOB1ya5n96s8qfR678+KMn2P8Zu4bpuBjZX1S3d+lX0\nkrC16wlwBvDdqtpaVT8FPkXvOo/qmjadJzAWuTIueQLjkytDzZMWClSzUyYlCXA5cHdVvXfaro3A\nqm55Fb3+9pGpqouq6rCqWkbv+t1YVecDNwEru8NaiPMR4MEkR3ebTgfuorHr2fkecHKSX+1+D7bH\nOqpr2myewHjkyrjkCYxVrgw3T0b9ZWD3JdqrgW8D3wHeOep4psX1MnpN6NuB27rHq+n1W98A3AN8\nDlg86linxfz7wLXd8m8BXwbuBT4B7NtAfMcDm7pr+h/Awa1eT+DdwDeBO4ArgH1HeU1bzZMutrHK\nldbzpItrLHJlmHniVEeSpCa10MUnSdIOLFCSpCZZoCRJTbJASZKaZIGSJDXJAjXmkpydpJIcM+pY\npFaZJ+PJAjX+zgO+0D1Lmpl5MoYsUGOsm/fsZfSmsj+32/acJJd195C5Psl/JlnZ7TsxyX8luTXJ\nddunTJEWMvNkfFmgxttZ9O4X823g8SQnAn8MLKN3z6A30Jvqfvs8af8IrKyqE4EPA383iqClPcw8\nGVOLZj9EDTuP3mSX0Jv88jx6/6afqKqfAY8kuanbfzTwQuD63pRZ7EVvenxpoTNPxpQFakwlWUxv\nNuYXJSl6iVTA1Ts7Bbizqk7ZQyFKI2eejDe7+MbXSuCKqvrNqlpWVYfTuwPn94HXdX3sS+hNigm9\nO3FOJfl5V0aSF4wicGkPMk/GmAVqfJ3Hjp8CPwn8Or17ydwF/BvwVeCpqvoJvWS9JMnX6c02/dI9\nF640EubJGHM28wUoyQFV9XSSQ+hNeX9q9e4vI6ljnrTP76AWpmuTHATsA/ytSSfNyDxpnC0oSVKT\n/A5KktQkC5QkqUkWKElSkyxQkqQmWaAkSU36f1L9Y6pVOfMFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x6c2cad0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "g = sns.FacetGrid(train_df, col='Survived')\n",
    "g.map(plt.hist, 'Age', bins=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 分析数值型和序数特征\n",
    "\n",
    "在一张图中分析多类别特征的相关性。\n",
    "\n",
    "### Observations\n",
    "1. Pclass=3的乘客最多，但是大多数都没有存活\n",
    "2. Pclass=2和Pclass=3的小孩大多数都存活\n",
    "3. Pclass=1的乘客大多存活\n",
    "4. Pclass在乘客的年龄分布方面各不相同。\n",
    "\n",
    "### Decisions\n",
    "把Pclass加入到模型中"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgAAAAHUCAYAAABMP5BeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X20ZHV95/v3h26IKCaIHLFDw4ABHxpmQOkgiiuXYIht\nzBUyQ1TGKMxFW9boGjQ+ANEkONEIY65oopMVLpBuvSoNgoFhMZAehKhRGxqheQzQIsbm0nSjEINx\nog3f+0ftxkNzDqdOnapzqmq/X2vVqtq//VDfX9X51vnWb+/aO1WFJElql50WOgBJkjT/LAAkSWoh\nCwBJklrIAkCSpBayAJAkqYUsACRJaiELAEmSWsgCoEdJHktyc5Lbklyc5JlPs+yZSd43n/FNE8eL\nk3wzyb8+XTxJViU5aor2vZJckWRDkjuSXNnH2M5LsqwP2zkpyaf7sJ3DktyaZGOSP0+SuW5Tw8t8\nHvt8/miS7yd5dK7bGicWAL37SVUdWlUHAz8FTlnogLrwQ+C/AH/W4/r/FVhbVYdU1TLg9NmsnGTR\ndPOq6m1VdUePcQ3CXwJvBw5sbisWNhwNmPk83vn8P4DDFzqIYWMB0B9fAw4ASPLWJLc0VfXndlww\nyduT3NDMv2T7N40kv9t8+9iQ5KtN20FJrm++mdyS5MC5BFlVW6rqBuBnPW5iCbBp0vZuaeI8KskV\n29uTfDrJSc3j+5KcneTbwPuTXD9puf2S3No8vi7J8iSnJPn4pGWe+AaQ5PcmvR5/tf0DKMl/SnJ3\ns+0je+zbE5IsAX6xqr5VnVNlfhY4bq7b1cgwn8con5u+fauqHujHtsaJBcAcJVkMvBa4NclBwIeA\no6vqEODUKVa5tKp+tZl/J3By0/5HwGua9tc3bacAn6qqQ4HlTErWSc+/pkmgHW9v7WtHOz4DnJ/k\n2iQfTPLLXa73g6p6WVWdBeySZP+m/Y3Amh2WvQT4nUnTbwQuTPKS5vGRzevxGPDm5p/1h+l8ULwK\nmHLYMcmvT/M6fWOKxffmya/1pqZNY8587sqo5bOmsXihAxhhuya5uXn8NeB84B3AxVX1EEBV/XCK\n9Q5O8hFgd2A34Oqm/e+BVUkuAi5t2r4JfDDJUjofNPfsuLGqemO/OjSTqro6yQvoDIe/FrgpycFd\nrDr5Q+EiOol/VnP/pPiramuSe5McAdwDvJjOa/NO4DDghnR2x+8KbAFeDlxXVVuh8wEKvHCK2K8F\nDu2+t2oZ89l8bh0LgN79pKlcn5DujhNbBRxXVRuaYbWjAKrqlCQvB14H3JjksKr6QpJ1TduVSd5R\nVV/Z4TnXAC+a4nk+UVWfnWWfZtR8CH4B+EIzTPhrwIM8eTTpGTus9uNJj9cAFye5tLO5p34IAhcC\nbwD+AfhyVVU6L+7qqjpj8oJJuhqaT/LrwDlTzPqXqnrlDm33A0snTS9t2jS+zOfxzWdNwwKgv74C\nfDnJJ6rqB0n2mOJbw7OBB5LsDLyZ5h9Lkl+pqnXAuiSvBfZJ8kvAvVX150n2Bf5d8xxPmM9vDEmO\nBr5VVf+S5NnArwD/CGwGliX5BTqV/KuBr0+1jar6TpLHgD/kqcOF230Z+CDwUuC0pu0a4LIk51TV\nliR70Hkt1wGfSvJc4EfA7wIbpnjerr8xVNUDSX7UfGtZB7wV+Itu1tVYMZ/HIJ81PQuAPqqq25N8\nFPi7JiluAk7aYbE/pPNHvrW5f3bT/vF0DgoKneTYQCdZ3pLkZ3SS8k/nEl+S5wPrgV8EHk/ybmBZ\nVf2oy00cBnw6yTY63xDOaw5CohnqvA34Lp1+P501wMeB/aeaWVUPJ7mzie36pu2OJB8C/jbJTnQO\nfHpnVX0ryZl0hlcfAW6eaps9+M90vt3tCvzP5qYWMZ/HJ5+T/DfgPwLPTLKJTl/P7Me2R1k6BzlL\nP5dkFbCqqq5b4FAkzZH5rOn4KwBJklrIAkBT+RvgvoUOQlJfmM+akrsAJElqIUcAJElqoXn9FcCK\nFSvqqquums+nlPRUfbmwkfksDY2ecnpeRwAeeuih+Xw6SQNkPkujzV0AkiS1kAWAJEktZAEgSVIL\nWQBIktRCFgCSJLWQBYAkSS1kASBJUgt1XQAkWZTkpiRXNNP7J1mXZGOSNUl2GVyYkiSpn2YzAnAq\ncOek6bOBc6rqAOBh4OR+BiZJkganqwIgyVLgdcB5zXSAo4EvNYusBo4bRICSJKn/uh0B+CTwAeDx\nZvq5wCNVta2Z3gTs3efYJEnSgMxYACT5bWBLVd3YyxMkWZlkfZL1W7du7WUTkoaE+SyNj25GAI4E\nXp/kPuBCOkP/nwJ2T7L9aoJLgfunWrmqzq2q5VW1fGJiog8hS1oo5rM0PmYsAKrqjKpaWlX7AW8C\nvlJVbwauBY5vFjsRuGxgUUqSpL6ay3kATgN+P8lGOscEnN+fkCRJ0qAtnnmRn6uq64Drmsf3Aof3\nPyRJkjRonglQkqQWsgCQJKmFLAAkSWohCwBJklrIAkCSpBayAJAkqYUsACRJaiELAEmSWsgCQJKk\nFrIAkCSphSwAJElqIQsASZJayAJAkqQWsgCQJKmFLAAkSWohCwBJklrIAkCSpBaasQBI8owk1yfZ\nkOT2JB9u2vdPsi7JxiRrkuwy+HAlSVI/dDMC8K/A0VV1CHAosCLJEcDZwDlVdQDwMHDy4MKUJEn9\nNGMBUB2PNpM7N7cCjga+1LSvBo4bSISSJKnvujoGIMmiJDcDW4C1wHeAR6pqW7PIJmDvwYQoSZL6\nrasCoKoeq6pDgaXA4cCLu32CJCuTrE+yfuvWrT2GKWkYmM/S+JjVrwCq6hHgWuAVwO5JFjezlgL3\nT7POuVW1vKqWT0xMzClYSQvLfJbGRze/AphIsnvzeFfgGOBOOoXA8c1iJwKXDSpISZLUX4tnXoQl\nwOoki+gUDBdV1RVJ7gAuTPIR4Cbg/AHGKUmS+mjGAqCqbgFeOkX7vXSOB5AkSSPGMwFKktRCFgCS\nJLWQBYAkSS1kASBJUgtZAEiS1EIWAJIktZAFgCRJLWQBIElSC1kASJLUQhYAkiS1kAWAJEktZAEg\nSVILWQBIktRCFgCSJLWQBYAkSS1kASBJUgtZAEiS1EIzFgBJ9klybZI7ktye5NSmfY8ka5Pc09w/\nZ/DhSpKkfuhmBGAb8N6qWgYcAbwzyTLgdOCaqjoQuKaZliRJI2DGAqCqHqiqbzeP/xm4E9gbOBZY\n3Sy2GjhuUEFKkqT+mtUxAEn2A14KrAP2qqoHmlmbgb36GpkkSRqYxd0umGQ34BLg3VX1oyRPzKuq\nSlLTrLcSWAmw7777zi1aSQvKfF5456y9u6vl3nPMCwcciUZdVyMASXam88//81V1adP8YJIlzfwl\nwJap1q2qc6tqeVUtn5iY6EfMkhaI+SyNj25+BRDgfODOqvrEpFmXAyc2j08ELut/eJIkaRC62QVw\nJPAW4NYkNzdtfwCcBVyU5GTge8AbBhOiJEnqtxkLgKr6OpBpZr+6v+FIkqT54JkAJUlqIQsASZJa\nyAJAkqQWsgCQJKmFLAAkSWohCwBJklrIAkCSpBayAJAkqYUsACRJaiELAEmSWqjrywFLkjq8JK/G\ngSMAkiS1kAWAJEktZAEgSVILWQBIktRCHgQoSWPIAxU1kxlHAJJckGRLktsmte2RZG2Se5r75ww2\nTEmS1E/d7AJYBazYoe104JqqOhC4ppmWJEkjYsZdAFX11ST77dB8LHBU83g1cB1wWh/jkqTW6Ha4\nflS4+2E09HoQ4F5V9UDzeDOwV5/ikSRJ82DOvwKoqgJquvlJViZZn2T91q1b5/p0khaQ+SyNj14L\ngAeTLAFo7rdMt2BVnVtVy6tq+cTERI9PJ2kYmM/S+Oi1ALgcOLF5fCJwWX/CkSRJ82HGgwCTfJHO\nAX97JtkE/DFwFnBRkpOB7wFvGGSQkjQf+n0w3rgd3Kfx0s2vAE6YZtar+xyLJEmaJ54JUPOum29F\n/jxIGi6OZowfrwUgSVILWQBIktRC7gJQXzlMKI0Wc7a9HAGQJKmFHAFQ63lQoqQ2cgRAkqQWsgCQ\nJKmF3AUw5OZreHoUh8FHMWYtDA90G079fl/M99lxBECSpBayAJAkqYXcBTAGZhpGc1hMkrQjRwAk\nSWohRwB60K+Dz0btwKRRixfmN2YPSpQ0ShwBkCSphSwAJElqIXcBDMgwDZcPUyzdGraYhy2eUdTt\na+huEvXKv7HZmdMIQJIVSe5KsjHJ6f0KSpIkDVbPIwBJFgGfAY4BNgE3JLm8qu7oV3DbeXCVpB05\nKqNRNEyjFHMZATgc2FhV91bVT4ELgWP7E5YkSRqkuRQAewPfnzS9qWmTJElDLlXV24rJ8cCKqnpb\nM/0W4OVV9a4dllsJrGwmXwTcNcOm9wQe6imo4TRO/RmnvkB7+/NQVa3o5Ql6yOfZxDUKxqkvYH+G\n2Wz60lNOz6UAeAVwZlW9ppk+A6CqPtbTBn++3fVVtXwu2xgm49SfceoL2J/5Mqxx9WKc+gL2Z5jN\nR1/msgvgBuDAJPsn2QV4E3B5f8KSJEmD1POvAKpqW5J3AVcDi4ALqur2vkUmSZIGZk4nAqqqK4Er\n+xTLduf2eXsLbZz6M059AfszX4Y1rl6MU1/A/gyzgfel52MAJEnS6PJaAJIktZAFgCRJLWQBIElS\nC1kASJLUQhYAkiS1kAWAJEktZAEgSVILWQBIktRCFgCSJLWQBYAkSS1kASBJUgtZAPQoyWNJbk5y\nW5KLkzzzaZY9M8n75jO+aeJ4c5Jbktya5BtJDplmuVVJjpqifa8kVyTZkOSOJH27EFSS85Is68N2\nTkry6T5s57DmddqY5M+TZK7b1PAyn8c+nz+a5PtJHp3rtsaJBUDvflJVh1bVwcBPgVMWOqAufBf4\nP6rq3wJ/wuyvNvVfgbVVdUhVLQNOn83KSRZNN6+q3lZVd8wynkH6S+DtwIHNbcXChqMBM5/HO5//\nB3D4QgcxbCwA+uNrwAEASd7aVOUbknxuxwWTvD3JDc38S7Z/00jyu823jw1Jvtq0HZTk+uabyS1J\nDpxLkFX1jap6uJn8FrB0lptYAmyatL1bmjiPSnLFpD5+OslJzeP7kpyd5NvA+5NcP2m5/ZLc2jy+\nLsnyJKck+fikZZ74BpDk9ya9Hn+1/QMoyX9Kcnez7SNn2aenSLIE+MWq+lZ1Lpf5WeC4uW5XI8N8\nHqN8bvr2rap6oB/bGicWAHOUZDHwWuDWJAcBHwKOrqpDgFOnWOXSqvrVZv6dwMlN+x8Br2naX9+0\nnQJ8qqoOBZYzKVknPf+aJoF2vL11htBPBv7nLLv7GeD8JNcm+WCSX+5yvR9U1cuq6ixglyT7N+1v\nBNbssOwlwO9Mmn4jcGGSlzSPj2xej8eANzf/rD9M54PiVcCUw45Jfn2a1+kbUyy+N09+rTc1bRpz\n5nNXRi2fNY3FCx3ACNs1yc3N468B5wPvAC6uqocAquqHU6x3cJKPALsDuwFXN+1/D6xKchFwadP2\nTeCDSZbS+aC5Z8eNVdUbZxt4kl+n84HxqtmsV1VXJ3kBneHw1wI3JTm4i1UnfyhcRCfxz2runxR/\nVW1Ncm+SI4B7gBfTeW3eCRwG3JDO7vhdgS3Ay4Hrqmpr07c1wAuniP1a4NDue6uWMZ/N59axAOjd\nT5rK9Qnp7jixVcBxVbWhGVY7CqCqTknycuB1wI1JDquqLyRZ17RdmeQdVfWVHZ5zDfCiKZ7nE1X1\n2R0bk/w74DzgtVX1g24Cnqz5EPwC8IVmmPDXgAd58mjSM3ZY7ceTHq8BLk5yaWdzT/0QBC4E3gD8\nA/Dlqqp0XtzVVXXGDv3pami++ZA8Z4pZ/1JVr9yh7X6ePJy6tGnT+DKfxzefNZ2q8tbDDXh0iraD\ngLuB5zbTezT3ZwLvax4/BDwP2BlYC6xq2n9l0nZuoFPdvgBI0/ZnwLvnGPO+wEbglTMstwo4aor2\no4FnNo+fTWfI81eBfYD7gF+g803ou8BJzXL3AXvusJ0bgM8BH5jUdh2wvHn8HOA7wLXA4U3bMjrf\nIJ63/bUF/g2d/ZjfA57bvKZfAz7dh/f3euAIIHSGVn9rof/mvA3uZj6Pdz4/3fvc5psjAH1UVbcn\n+Sjwd0keA24CTtphsT8E1gFbm/tnN+0fbw4KCnANsAE4DXhLkp8Bm4E/nWOIf0Qnsf578+1mW1Ut\nn8X6hwGfTrKNzjeE86rqBoBmqPM2Oh8WN82wnTXAx4H9p5pZVQ8nuRNYVlXXN213JPkQ8LdJdgJ+\nBryzqr6V5Ew6w6uPADdPtc0e/Gc6H5y70ikAZrt/VSPOfB6ffE7y34D/CDwzySY6fT2zH9seZdur\nUekJSVbR+SZz3QKHImmOzGdNx18BSJLUQhYAmsrf0NnXJ2n0mc+akrsAJElqIUcAJElqIQsASZJa\naF5/BrhixYq66qqr5vMpJT1VX65saD5LQ6OnnJ7XEYCHHnpoPp9O0gCZz9JocxeAJEktZAEgSVIL\nWQBIktRCFgCSJLWQBYAkSS1kASBJUgtZAEiS1EJdFwBJFiW5KckVzfT+SdYl2ZhkTZJdBhemJEnq\np9mMAJwK3Dlp+mzgnKo6AHgYOLmfgUmSpMHpqgBIshR4HXBeMx3gaOBLzSKrgeMGEaAkSeq/bkcA\nPgl8AHi8mX4u8EhVbWumNwF79zk2SZI0IDMWAEl+G9hSVTf28gRJViZZn2T91q1be9mEpCFhPkvj\no5sRgCOB1ye5D7iQztD/p4Ddk2y/muBS4P6pVq6qc6tqeVUtn5iY6EPIkhaK+SyNjxkLgKo6o6qW\nVtV+wJuAr1TVm4FrgeObxU4ELhtYlJIkqa/mch6A04DfT7KRzjEB5/cnJEmSNGiLZ17k56rqOuC6\n5vG9wOH9D0mSJA2aZwKUJKmFLAAkSWohCwBJklrIAkCSpBayAJAkqYUsACRJaiELAEmSWsgCQJKk\nFrIAkCSphSwAJElqIQsASZJayAJAkqQWsgCQJKmFLAAkSWohCwBJklrIAkCSpBaasQBI8owk1yfZ\nkOT2JB9u2vdPsi7JxiRrkuwy+HAlSVI/dDMC8K/A0VV1CHAosCLJEcDZwDlVdQDwMHDy4MKUJEn9\nNGMBUB2PNpM7N7cCjga+1LSvBo4bSISSJKnvujoGIMmiJDcDW4C1wHeAR6pqW7PIJmDvwYQoSZL6\nrasCoKoeq6pDgaXA4cCLu32CJCuTrE+yfuvWrT2GKWkYmM/S+JjVrwCq6hHgWuAVwO5JFjezlgL3\nT7POuVW1vKqWT0xMzClYSQvLfJbGRze/AphIsnvzeFfgGOBOOoXA8c1iJwKXDSpISZLUX4tnXoQl\nwOoki+gUDBdV1RVJ7gAuTPIR4Cbg/AHGKUmS+mjGAqCqbgFeOkX7vXSOB5AkSSPGMwFKktRCFgCS\nJLWQBYAkSS1kASBJUgtZAEiS1EIWAJIktZAFgCRJLWQBIElSC1kASJLUQhYAkiS1kAWAJEktZAEg\nSVILWQBIktRCFgCSJLWQBYAkSS1kASBJUgtZAEiS1EIzFgBJ9klybZI7ktye5NSmfY8ka5Pc09w/\nZ/DhSpKkfuhmBGAb8N6qWgYcAbwzyTLgdOCaqjoQuKaZliRJI2DxTAtU1QPAA83jf05yJ7A3cCxw\nVLPYauA64LSBRClJkp7WjTfe+LzFixefBxzMk7/gPw7ctm3btrcddthhW7Y3zlgATJZkP+ClwDpg\nr6Y4ANgM7DWHuCVJ0hwsXrz4vOc///kvmZiYeHinnXaq7e2PP/54tm7dumzz5s3nAa/f3t71QYBJ\ndgMuAd5dVT+aPK+qCqhp1luZZH2S9Vu3bp1tfyQNEfNZGmoHT0xM/GjyP3+AnXbaqSYmJv6JzsjA\nz9u72WKSnen88/98VV3aND+YZEkzfwmwZap1q+rcqlpeVcsnJiZm2RdJw8R8lobaTjv+8580o9jh\nf343vwIIcD5wZ1V9YtKsy4ETm8cnApf1FK4kSZp33RwDcCTwFuDWJDc3bX8AnAVclORk4HvAGwYT\noiRJ6rdufgXwdSDTzH51f8ORJEk9evzxxx/PVLsBHn/88dD5NcATPBOgJEnj4batW7f+UvPP/gnN\nrwB+CbhtcvusfgYoSZKG07Zt2962efPm8zZv3jzteQAmL28BIEnSGGhO8vP6GRdsuAtAkqQWsgCQ\nJKmFLAAkSWohCwBJklrIAkCSpBayAJAkqYUsACRJaiELAEmSWsgCQJKkFrIAkCSphSwAJElqIQsA\nSZJayAJAkqQWsgCQJKmFZiwAklyQZEuS2ya17ZFkbZJ7mvvnDDZMSZLUT4u7WGYV8Gngs5PaTgeu\nqaqzkpzeTJ/W//AEcM7au6ed955jXjiPkUiSxsWMIwBV9VXghzs0Hwusbh6vBo7rc1ySJGmAej0G\nYK+qeqB5vBnYq0/xSJKkedDNLoCnVVWVpKabn2QlsBJg3333nevTjbRBDOW7e0Dzqdd8frq/U/Bv\ndSbmuQah1xGAB5MsAWjut0y3YFWdW1XLq2r5xMREj08naRiYz9L46HUE4HLgROCs5v6yvkUkSSPG\nEQ6Nom5+BvhF4JvAi5JsSnIynX/8xyS5B/iNZlqSJI2IGUcAquqEaWa9us+xSJKkeTLngwA1mjyo\nSBoeM+1CkAbBUwFLktRCjgAMiUF8A/BbhdSdQR/EZy5qGDkCIElSC1kASJLUQu4C6IHDeZKkUecI\ngCRJLWQBIElSC1kASJLUQhYAkiS1kAcB6ik8S6DmmxfT6d1cXztf+/ZyBECSpBayAJAkqYXcBaC+\n6fX8CL0OMc7382nhLPQw9Sif+2OhY1/o907TcwRAkqQWcgRAs+JFizSK/Bvr3UJfKMkRgsGZ0whA\nkhVJ7kqyMcnp/QpKkiQNVs8FQJJFwGeA1wLLgBOSLOtXYJIkaXDmsgvgcGBjVd0LkORC4Fjgjn4E\nJg1Kr+c58KBD6akWeveKuxB6N5ddAHsD3580valpkyRJQy5V1duKyfHAiqp6WzP9FuDlVfWuHZZb\nCaxsJl8E3DXDpvcEHuopqOE0Tv0Zp75Ae/vzUFWt6OUJesjn2cQ1CsapL2B/htls+tJTTs+lAHgF\ncGZVvaaZPgOgqj7W0wZ/vt31VbV8LtsYJuPUn3HqC9if+TKscfVinPoC9meYzUdf5rIL4AbgwCT7\nJ9kFeBNweX/CkiRJg9TzQYBVtS3Ju4CrgUXABVV1e98ikyRJAzOnEwFV1ZXAlX2KZbtz+7y9hTZO\n/RmnvoD9mS/DGlcvxqkvYH+G2cD70vMxAJIkaXR5LQBJklrIAkCSpBayAJAkqYUsACRJaiELAEmS\nWsgCQJKkFrIAkCSphSwAJElqIQsASZJayAJAkqQWsgCQJKmFLAAkSWohC4AeJXksyc1JbktycZJn\nPs2yZyZ533zGN00cxya5pYl7fZJXTbPcdUn2m6L9Rc28m5PcmaRvV6tKcmWS3fuwnb681klWJLkr\nycYkp891expu5vPY5/MFSbYkuW2u2xonFgC9+0lVHVpVBwM/BU5Z6IC6cA1wSFUdCvxfwHmzXP/P\ngXOafr8E+IvZrJxk0XTzquq3quqRWcYzEE2cnwFeCywDTkiybGGj0oCZz2Oaz41VwIqFDmLYWAD0\nx9eAAwCSvLWpyjck+dyOCyZ5e5IbmvmXbP+mkeR3m28fG5J8tWk7KMn1TYV+S5ID5xJkVT1aP7/+\n87OA2V4LegmwadL2bm3iPCnJpyf18YokRzWPH03yfyfZAJyR5OJJyx2V5Irm8X1J9kxyVpJ3Tlrm\niW8ASd7fvHa3JPnwpGU+mOTuJF8HXjTLPk3lcGBjVd1bVT8FLgSO7cN2NRrM5/HKZ6rqq8AP+7Gt\ncbJ4oQMYdUkW0/mmeFWSg4APAa+sqoeS7DHFKpdW1f/TrPsR4GQ6lfcfAa+pqvsnDZ2dAnyqqj6f\nZBfgKRV3kjVMnSSfqKrPTrH87wAfA54HvG6W3T0H+EqSbwB/C/x1F1X+s4B1VfXe5rW6N8mzqurH\nwBvp/HOdbA3wSTrfwAHeALwmyW8CB9L55xzg8iS/BvwYeBNwKJ2/528DN+4YRJI3A++fIr6NVXX8\nDm17A9+fNL0JePkM/dQYMJ/HMp81DQuA3u2a5Obm8deA84F3ABdX1UMAVTVVxXlw80GxO7AbcHXT\n/vfAqiQXAZc2bd8EPphkKZ0Pmnt23FhVvXE2QVfVl4EvN8n2J8BvzGLdv05yNZ2htGOBdyQ5ZIbV\nHgMuadbfluQq4P9M8iU6H1gf2OE5bkryvCS/DEwAD1fV95OcCvwmcFOz6G50PkCeDXy5qv4FIMnl\n08T+eeDz3fZVrWM+m8+tYwHQu580+96ekKSb9VYBx1XVhiQnAUcBVNUpSV5OJ4luTHJYVX0hybqm\n7cok76iqr+zwnLP6xrBdVX01yQuS7Ln9A64bVfX/ARcAF6RzQM3BwDaevDvpGZMe/++qemzS9IXA\nu+gMx62vqn+e4mkuBo4Hnk/nGwR0viV8rKr+avKCSd7dTdyz/MZwP7DPpOmlTZvGl/k8vvms6VSV\ntx5uwKNTtB0E3A08t5neo7k/E3hf8/ghOsN1OwNrgVVN+69M2s4NdIbAXgCkafsz4N1zjPmASdt7\nGZ1/apliueuA/aZoXwHs3Dx+PvBAc/8q4Bt0PjT2AX4EHDXV60Rn2PM+Oh8Kb5jUfh+w56TX8RvN\na7mkaftNYB2wWzO9d/M6vgy4BdiVzreHe7a/1nN4nRYD9wL7A7sAG4CDFvpvztvgbubz+ObzpJj2\nA25b6L+1Ybo5AtBHVXV7ko8Cf5fkMTrDWyftsNgf0vnD39rcP7tp/3hzUFDoHN27ATgNeEuSnwGb\ngT+dY4j/AXhrs72fAG+sJjO69JvAp5L872b6/VW1OcmDwHeBO4A76ey3m1JVPdYcKHQScOI0y9ye\n5NnA/VX1QNP2t0leAnyz+Wb2KPB7VfXt5lvTBmALnQ/bOanO0Oa76AznLgIuqKrb57pdjRbzeTzy\nGSDJF+mMzuyZZBPwx1V1fj+2Pcoyu78XtUGS64CTquq+BQ5F0hyZz5qOPwOUJKmFLAA0lVXAMJ3E\nQ1LvVmG9za+BAAAMzElEQVQ+awruApAkqYUcAZAkqYXm9VcAK1asqKuuumo+n1LSU3X1A/eZmM/S\n0Ogpp+d1BOChh7o+P4WkIWc+S6PNXQCSJLWQBYAkSS1kASBJUgtZAEiS1EJeC2BAzll794zLvOeY\nF85DJJIkPZUjAJIktZAFgCRJLdR1AZBkUZKbmks/kmT/JOuSbEyyJskugwtTkiT102xGAE6lc23o\n7c4GzqmqA4CHgZP7GZgkSRqcrgqAJEuB1wHnNdMBjga+1CyyGjhuEAFKkqT+63YE4JPAB4DHm+nn\nAo9U1bZmehOwd59jkyRJAzJjAZDkt4EtVXVjL0+QZGWS9UnWb926tZdNSBoS5rM0ProZATgSeH2S\n+4AL6Qz9fwrYPcn28wgsBe6fauWqOreqllfV8omJiT6ELGmhmM/S+JixAKiqM6pqaVXtB7wJ+EpV\nvRm4Fji+WexE4LKBRSlJkvpqLmcCPA24MMlHgJuA8/sT0nDr5gx/kiQNu1kVAFV1HXBd8/he4PD+\nhyRJkgbNMwFKktRCFgCSJLWQBYAkSS1kASBJUgtZAEiS1EIWAJIktdBczgOgOer2nALvOeaFA45E\nktQ2jgBIktRCFgCSJLWQBYAkSS1kASBJUgtZAEiS1EIWAJIktZAFgCRJLWQBIElSC1kASJLUQp4J\ncAR0c8ZAzxYoSZqNGUcAkjwjyfVJNiS5PcmHm/b9k6xLsjHJmiS7DD5cSZLUD93sAvhX4OiqOgQ4\nFFiR5AjgbOCcqjoAeBg4eXBhSpKkfpqxAKiOR5vJnZtbAUcDX2raVwPHDSRCSZLUd10dBJhkUZKb\ngS3AWuA7wCNVta1ZZBOw92BClCRJ/dZVAVBVj1XVocBS4HDgxd0+QZKVSdYnWb9169Yew5Q0DMxn\naXzM6meAVfUIcC3wCmD3JNt/RbAUuH+adc6tquVVtXxiYmJOwUpaWOazND66+RXARJLdm8e7AscA\nd9IpBI5vFjsRuGxQQUqSpP7q5jwAS4DVSRbRKRguqqorktwBXJjkI8BNwPkDjFOSJPXRjAVAVd0C\nvHSK9nvpHA8gSZJGjKcCliSphSwAJElqIQsASZJayAJAkqQWsgCQJKmFLAAkSWohCwBJklrIAkCS\npBayAJAkqYUsACRJaiELAEmSWsgCQJKkFrIAkCSphSwAJElqIQsASZJayAJAkqQWsgCQJKmFZiwA\nkuyT5NokdyS5PcmpTfseSdYmuae5f87gw5UkSf3QzQjANuC9VbUMOAJ4Z5JlwOnANVV1IHBNMy1J\nkkbAjAVAVT1QVd9uHv8zcCewN3AssLpZbDVw3KCClCRJ/TWrYwCS7Ae8FFgH7FVVDzSzNgN79TUy\nSZI0MIu7XTDJbsAlwLur6kdJnphXVZWkpllvJbASYN99951btJrWOWvv7mq59xzzwgFHonFmPkvj\no6sRgCQ70/nn//mqurRpfjDJkmb+EmDLVOtW1blVtbyqlk9MTPQjZkkLxHyWxseMIwDpfNU/H7iz\nqj4xadblwInAWc39ZQOJUPOum9EERxIkabR1swvgSOAtwK1Jbm7a/oDOP/6LkpwMfA94w2BClCRJ\n/TZjAVBVXwcyzexX9zccSZI0HzwToCRJLWQBIElSC3X9M0CNh25/LihJGm+OAEiS1EIWAJIktZAF\ngCRJLWQBIElSC1kASJLUQhYAkiS1kAWAJEktZAEgSVILWQBIktRCFgCSJLWQBYAkSS1kASBJUgsN\n9cWAurlwzXuOeeE8RCJJ0niZcQQgyQVJtiS5bVLbHknWJrmnuX/OYMOUJEn91M0ugFXAih3aTgeu\nqaoDgWuaaUmSNCJm3AVQVV9Nst8OzccCRzWPVwPXAaf1Ma6+6mZXArg7QZLUHr0eBLhXVT3QPN4M\n7NWneCRJ0jyY80GAVVVJarr5SVYCKwH23XffuT6dhkS3oyrdcORldLQ1n2f6e5/pb3iu60uD0OsI\nwINJlgA091umW7Cqzq2q5VW1fGJiosenkzQMzGdpfPRaAFwOnNg8PhG4rD/hSJKk+TDjLoAkX6Rz\nwN+eSTYBfwycBVyU5GTge8AbBhmkpNHmEPjT8/XRQujmVwAnTDPr1X2ORZIkzZOhPhOgpHYY9Dfg\nfh60Ko0LrwUgSVILWQBIktRC7gKQpDlyF4NGkSMAkiS1kCMAkjTkvDS6BsERAEmSWsgCQJKkFhr5\nXQD9PPjGA3kWhpdrlqT55wiAJEktZAEgSVILWQBIktRCFgCSJLWQBYAkSS1kASBJUgtZAEiS1EIj\nfx4AaRR4Kte5men187Wb+3lMfA3bZ04jAElWJLkrycYkp/crKEmSNFg9jwAkWQR8BjgG2ATckOTy\nqrqjX8FJkw3jmRr91jQchvFvY9SM+giCo0SzN5cRgMOBjVV1b1X9FLgQOLY/YUmSpEGaSwGwN/D9\nSdObmjZJkjTkBn4QYJKVwMpm8tEkd82wyp7AQ4ONal6NU3/GqS/Qh/78fp8C6dO2uu3PVVW1opcn\n6CGfZxPXKBinvkAf+9PPXJiDafszJPHNxmzem55yOlU123U6KyavAM6sqtc002cAVNXHetrgz7e7\nvqqWz2Ubw2Sc+jNOfQH7M1+GNa5ejFNfwP4Ms/noy1x2AdwAHJhk/yS7AG8CLu9PWJIkaZB63gVQ\nVduSvAu4GlgEXFBVt/ctMkmSNDBzOgagqq4EruxTLNud2+ftLbRx6s849QXsz3wZ1rh6MU59Afsz\nzAbel56PAZAkSaPLawFIktRCQ1UAjPKphZPsk+TaJHckuT3JqU37HknWJrmnuX/OQsc6G0kWJbkp\nyRXN9P5J1jXv0ZrmANCRkGT3JF9K8g9J7kzyilF9f5K8p/k7uy3JF5M8Y9jem1HOZxjPnDafh9dC\n5PTQFACTTi38WmAZcEKSZQsb1axsA95bVcuAI4B3NvGfDlxTVQcC1zTTo+RU4M5J02cD51TVAcDD\nwMkLElVvPkXn97IvBg6h06+Re3+S7A38F2B5VR1M5yDcNzFE780Y5DOMZ06bz0NowXK6qobiBrwC\nuHrS9BnAGQsd1xz6cxmd6yTcBSxp2pYAdy10bLPow1I6SXQ0cAUQOiemWDzVezbMN+CXgO/SHPcy\nqX3k3h9+fhbOPegcyHsF8Jphem/GLZ+bPox0TpvPw3tbqJwemhEAxujUwkn2A14KrAP2qqoHmlmb\ngb0WKKxefBL4APB4M/1c4JGq2tZMj9J7tD+wFfjrZgj0vCTPYgTfn6q6H/gz4B+BB4B/Am5kuN6b\nsclnGJucNp+H1ELl9DAVAGMhyW7AJcC7q+pHk+dVp4wbiZ9dJPltYEtV3bjQsfTJYuBlwF9W1UuB\nH7PD8OCovD/Nfs1j6XwI/jLwLKCnU/tqZuOQ0+bzcFuonB6mAuB+YJ9J00ubtpGRZGc6HxSfr6pL\nm+YHkyxp5i8BtixUfLN0JPD6JPfRudLj0XT2ue2eZPv5I0bpPdoEbKqqdc30l+h8gIzi+/MbwHer\namtV/Qy4lM77NUzvzcjnM4xVTpvPw21BcnqYCoCRPrVwkgDnA3dW1ScmzbocOLF5fCKd/YhDr6rO\nqKqlVbUfnffiK1X1ZuBa4PhmsVHqz2bg+0le1DS9GriD0Xx//hE4Iskzm7+77X0ZpvdmpPMZxiun\nzeehtzA5vdAHP+xwIMRvAXcD3wE+uNDxzDL2V9EZbroFuLm5/Rad/WzXAPcA/wvYY6Fj7aFvRwFX\nNI9fAFwPbAQuBn5hoeObRT8OBdY379HfAM8Z1fcH+DDwD8BtwOeAXxi292aU87mJfyxz2nwezttC\n5LRnApQkqYWGaReAJEmaJxYAkiS1kAWAJEktZAEgSVILWQBIktRCFgB6kiTHJakkL17oWCTNnTmt\n6VgAaEcnAF9v7iWNPnNaU7IA0BOac56/is4lJ9/UtO2U5L8319xem+TKJMc38w5L8ndJbkxy9fZT\ncEoaDua0no4FgCY7ls71te8GfpDkMODfA/vRuab7W+hcknL7OdL/Aji+qg4DLgA+uhBBS5qWOa1p\nLZ55EbXICXQuEAKdC4acQOdv5OKqehzYnOTaZv6LgIOBtZ1TV7OIzmUsJQ0Pc1rTsgAQAEn2oHOF\nsH+bpOgkfwFfnm4V4PaqesU8hShpFsxpzcRdANrueOBzVfVvqmq/qtoH+C7wQ+A/NPsN96JzIRGA\nu4CJJE8MHyY5aCEClzQlc1pPywJA253AU78ZXAI8n861t+8A/l/g28A/VdVP6XzAnJ1kA50rpb1y\n/sKVNANzWk/LqwFqRkl2q6pHkzyXzqUpj6zO9bgljSBzWuAxAOrOFUl2B3YB/sQPCmnkmdNyBECS\npDbyGABJklrIAkCSpBayAJAkqYUsACRJaiELAEmSWsgCQJKkFvr/AR5CjtONRzOHAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x6fe4950>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# grid = sns.FacetGrid(train_df, col='Pclass', hue='Survived')\n",
    "grid = sns.FacetGrid(train_df, col='Survived', row='Pclass', size=2.2, aspect=1.6)\n",
    "grid.map(plt.hist, 'Age', alpha=.5, bins=20)\n",
    "grid.add_legend();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 类别特征的相关性\n",
    "分析类别特征和存活目标的相关性\n",
    "\n",
    "### Observations\n",
    "1. Female乘客的存活率比males高\n",
    "2. 其中Embarked=C是一个例外，male有更高的存活率。这可能是Pclass和Embarked之间的相关性，反过来也是Pclass和Survived，并不一定是直接相关的\n",
    "3. C和Q上船的Pclass=3中male存活率更高\n",
    "4. 在Pclass=3的male中，不同上船口的存活率不同\n",
    "\n",
    "### Decisions\n",
    "1. 把Sex加入模型中\n",
    "2. 补全并把Embarked加入到模型中"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/lib/python2.7/site-packages/seaborn/axisgrid.py:703: UserWarning: Using the pointplot function without specifying `order` is likely to produce an incorrect plot.\n",
      "  warnings.warn(warning)\n",
      "/usr/lib/python2.7/site-packages/seaborn/axisgrid.py:708: UserWarning: Using the pointplot function without specifying `hue_order` is likely to produce an incorrect plot.\n",
      "  warnings.warn(warning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x7c46990>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATsAAAHUCAYAAABFzo+QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl8VNX5+PHPM5N9YU2AsAmy71tEQCygaNFa/bV1rW21\natHWar+1/bpUqnWrWqtWuol1wW+tC4XW4lJRBFwQkLDvO0hYQwIBsk/m+f1xb8IkZJmETCbJPO/X\na16Tu5x7nxvjw7n3nHuOqCrGGNPSecIdgDHGNAZLdsaYiGDJzhgTESzZGWMigiU7Y0xEsGRnjIkI\nluyaKREpFZHVAZ9761B2ooi8e4bnXyQi6fUse8bnd49zmYisEpE1IrJRRG4902Oalisq3AGYeitQ\n1eHhOLGIeMNx3koxRAMvAKNVNVNEYoEe4Y3KNGVWs2thRGS3iDzu1vYyRGSkiMwTkR0iclvArq1E\n5D0R2SIiz4uIxy3/V7fcBhF5qNJxnxSRlcBVAes9IjJTRB51ly8WkSUislJE/ikiSe76KSKy2S3/\n7Qa41GScf6yzAVS1SFW3NMBxTQtlya75iq90G3tNwLav3FrfZ8BM4EpgDPBQwD6jgTuAgUAvTiWg\n+1U1HRgKTBCRoQFlslV1pKq+6S5HAf8AtqnqNBFJAaYBk1V1JJAB3CUiccDfgG8Co4BOVV2QiPSr\ndE2BnzaB+6pqDjAX2CMib4jI9WUJ25iq2G1s81XTbexc93sdkKSqJ4ATIlIUkDS+VNWdACLyBjAe\nmA1cLSJTcf420nCS4Vq3zFuVzjMDmKWqj7nLY9z9F4sIQAywBOgP7FLVbe75XgOmVg7arZkFfWuu\nqreIyBBgMvBL4CLgxmDLm8hiya5lKnK//QE/ly2X/Tev/FK0ikhPnKRxjqoeFZGZQFzAPnmVynwB\nTBKRp1W1EBDgI1W9LnAnEQkqgYlIP05PqGUmquqxyitVdR2wTkT+DuzCkp2phlX7I9doEenp3vpd\nA3wOtMJJaLki0hG4pJZjvAS8D8wSkShgKXCeiPQGEJFEEekLbAZ6iEgvt9x1VR1MVbeo6vBqPhUS\nnYgkicjEgFXDgT11uH4TYaxm13zFi8jqgOUPVDXo7ifAcuBPQG9gIfBvVfWLyCqc5LQXWFzbQVT1\nGRFpDfwduB6nZvWG2zoKME1Vt7q3xu+JSD7Os8TkOsRaFQHuFpEZQAFOkr7xDI9pWjCxIZ6MMZHA\nbmONMRHBkp0xJiJYsjPGRARLdsaYiGDJzhgTEVpMspsyZYridJS1j30i6WOC1GKS3ZEjR8IdgjGm\nCWsxyc4YY2piyc4YExEs2RljIkLIkp2IvCwih0VkfTXbRUSmi8h2EVkrIiMDtt0gItvczw2hitEY\nEzlCWbObCUypYfslQB/3MxX4K4CItAMeBM7FGWDyQRFpG8I4jTERIGTJTlU/BXJq2OUK4P/UsRRo\nIyJpwNdxxkTLUdWjwEfUnDSNMaZW4RziqQvOMEJlMt111a1vcnz+UpbvW82rq2ZT6CskNaE9j198\nH1GesM9HY4yppFk3UIjIVHdymIysrKxGPXdOwTHu+fC3PPvFi+QUHCO/pJA9ufu478PHOVaQ26ix\nGGNqF85ktw/oFrDc1V1X3frTqOoLqpququmpqakhC7Qqzy15mb25+09bvyd3H9OXvtKosRhjahfO\nZDcX+IHbKjsGyFXVA8A84GIRaes2TFzsrmsydh3dy6asbdVuX394Cwt2LuarY/s4WpCLr9TXiNE1\nrkcXTedn7z3Io4umhzsUY2oUsmd27oxVE4EUEcnEaWGNBlDV53HmLrgU2A7kAz90t+WIyCM4w4YD\nPOxOm9dk7Dq6t9Z9nl/+WoXl+Kg4kmMTSY5JIjk2kaTYJFrFJJIc6ywnxyaRXLbs7hPtjQ7VJTSY\nrLxsDpw8HO4wjKlVyJJd5RmmqtiuwO3VbHsZeDkUcTWEuKjY2neqpMBXSIGvkMN52XU6T3JgQoxJ\nIik2kVYBCbFikkwkJiqmzrHVx4mik3y04zOO5B8F4HjRSbLysklNbN8o5zemrmzCnXoYnjaQWG8M\nRaXFVW6P9kbx7QFTKPAVcaIojxNFJzlR7H4XneRkcT4axIAVhb4iCn1FZOUHX7GN9cZUSIBJsYm0\nCkyS5bXLU/vE1jFBHjxxmIcWPkt2wakJv/JK8vnFB49w7/m3M7BDnzodz5jG0GIm3ElPT9eMjIxG\nO9/7Wxcwc9U/q9x208hrmNJnYrVl/X4/eSX5AQmwLCGePD05BnyH6r9VtDeaVuU1xYo1yKSYsu8k\nWrk1yeeWvMTW7F1VHqttXGv+fNmjRHnt39FGIuEOoLmwv8h6urTvBSRGJzBn4/scPOl0e/GKlx+P\n/j5f63FujWU9Ho97a5oEyR2DOp9f/eSXFFRRU8yrMUn61V/rsUtKS8guOEp2wdGgYqnJ0cJcMvav\nZUy3kbXvbEwjsmR3Bib0HMP5PUZzx3sPlD+vqi3R1ZdHPCTFJJIUk0hacoegyqgqBSWFHC8+WakG\nefqtdeC6Un/pGcValvyNaUos2Z0hj3iIEueNiaZ2PyEiJMTEkxATT6ek4PohqioFvkJOFuVxvOgk\nJ4ud77JEmJl7gC/3ra7xGG3jWjdE+MY0KEt2pgIRISE6noToeDokpZy2XVX5xQePkHn8QLXHOF50\nMpQhGlMvzfp1saYiNbE9aUkdIqLbhYgwNf16Yr3Vt+D+fc0cZq6chd9f+/NCYxqLtcaaevnq2D7e\n3vwhi/csR1FivTFMOnsc87Z/Ut5qPKrzEH425ibiouPCHG2L1tSenjRZVrMz9dK9TRfuHPPD8meB\n7eLbcNPIa7h7/I+JdTtdr9i/jgcXPENO/rGaDmVMo7BkZxrUqM5DePiCX9Auvg0Au47t5b75TwT1\nip0xoWTJzjS4nm278dvJ99CzjTN4zdGCXB5Y8DQZ+9aGOTITySzZmZBol9CGhy64i1GdhwBQ5Cvi\nqc+f5/2tC0L2JogxNbFkZ85ITS3RcdFx/O95t/GNvhcCoCgzV/2Tl1e+dcYdl42pK2uNNY3iw+2f\n8NLKt8prdSPSBvGzsTeTEB0f5siaPWuNDZLV7EyjuLj3BO47/3bio5xuKKsObOCBj5/mSF6TGqrQ\ntGCW7EyjGZ42iEcu/CXtE5yZMb/K3cev5j/Jjpw9YY7MRIKQJjsRmSIiW9yJsO+tYvuzIrLa/WwV\nkWMB20oDts0NZZym8XRv04XfTr6HXm3PAuBY4XEeXPA0X2bW/L6tMWcqZM/sRMQLbAUuwpkOcTlw\nnapurGb/O4ARqnqTu3xSVZOCPZ89s2teinzF/HHZK+VJThCuH/YtvtlvMiL2GKoO7JcVpFDW7EYD\n21V1p6oWA2/iTIxdneuAN0IYj2lCYqNiuGvcj7i8/0WA01L72pp/8beM1/FZS60JgVAmu6AnuxaR\ns4CewIKA1XHunLBLReT/hS5MEy4e8fC9Yd9mavr1eMT5U5y/83Oe+PTP5BcXhDk609I0lQaKa4HZ\nqhr4T/pZqpoOfBf4g4j0qlwonJNkm4Yzudd4fvW1nxLvDhiw9tAmpn38VJ0mJzKmNqFMdkFPdo2T\n7CrcwqrqPvd7J7AIGFG5UDgnyTYNa2inATx24d3lnZMzjx/g/o+eZFs1c10YU1ehTHbLgT4i0lNE\nYnAS2mmtqiLSH2gLLAlY11ZEYt2fU4DzgCobNkzL0bV1Go9Nvps+7XsCkFt0gt8sfJYle1eEOTLT\nEoQs2amqD/gpMA/YBMxS1Q0i8rCIXB6w67XAm1qxWXgAkCEia4CFwBPVteKalqVNXCsenPg/jO02\nCnAmA3r2ixf598YP7J1ac0bsdTHTJPnVz1vr3uHfmz4oXzep5zh+NOo6m6axIut6EqSm0kBhTAUe\n8XDd0Cv48Tnfx+u21C7c9QWPffpHThbnhTk60xzVmOxE5ISIHK/u01hBmsg16exx3D/hThLdAQM2\nHN7KtPlP2XSNps5qTHaqmqyqrYDngHtx+sl1Be4B/hD68IyBwR378djku+noDgG//8Qh7p//OzZn\n7QhzZKY5CeqZnYisUdVhta0LJ3tm1/IdLzrJU58/z5YjTpKL8kTxk9HfZ/xZo8McWVjZM7sgBfvM\nLk9ErhcRr4h4ROR6wB6cmEbVKjaJX0/8GeO7nwOAz+9j+tJXmL3hPWupNbUKNtl9F7gaOOR+rnLX\nGdOoYrzR3DHmh1w56Bvl62atf5c/L3uVktKSMEZmmjrremKarU93L+P55a/h8/sAGJDam1+edyvJ\nsUEPltMS2G1skIKq2YlIXxH5WETWu8tDRWRaaEMzpmZf63Euv554J8kxiQBsytrO/fN/x/4Th8Ic\nmWmKgr2N/RtwH1ACoKprcd58MCasBqT24bHJd5OW1AGAgyezmDb/KTYe3hrmyExTE2yyS1DVLyut\n8zV0MMbUR6fkDjw2+W4GpvYB4GRxHo98Mp1Pdi0Nc2SmKQk22R1xh1hSABG5EjgQsqiMqaOk2ESm\nTbiTr/U4F4BSfyl//vJV3lr3jrXUGiD4ZHc7MAPoLyL7gP8BbgtZVMbUQ5Q3ittH38C1Q06NMzFn\n4/s8t/Rliq2lNuIF+0b1HlWdLCKJgEdVT4QyKGPqS0T49sBL6JiUwl+W/R8lfh9ffJVBdl4O/zv+\nNlrFJYc7RBMmwdbsdonIC8AY4GQI4zGmQZzX/RwemPQ/5d1QtmTv5FfznyTzuD19iVTBJrv+wHyc\n29ldIvInERkfurCMOXP9Unrx28l30yW5EwCH87KZNv8p1h/aHObITDgElexUNV9VZ6nqt3GGR28F\nfBLSyIxpAB2TUnl08v8yuEM/APJLCnjskz+yYOcXYY7MNLagx7MTkQki8hdgBRCH8/pYbWVqmyT7\nRhHJCpgM+5aAbTeIyDb3c0OwcRpTWWJMAr+acAcX9BwHQKn6eX7533l97dv41R/m6ExjCXbUk93A\nKmAWMFdVax0EIJhJskXkRiBdVX9aqWw7IANIx+nusgIYpapHqzufvS5maqOqzN38Ef9Y++/ydWO6\njuSn595ATFRMGCM7I/a6WJCCrdkNVdVvqeobwSQ6V10nyQ70deAjVc1xE9xHwJQgyxpTJRHhigEX\nc9e4HxHtjQZgaeZKfrPwWY4V2li0LV1tIxXf7f74mIhMr/yp5djBTpL9HRFZKyKzRaRs6sWgJ9g2\npq7GdBvJQ5PuonVcKwC25+zm/o+eZG/u/jBHZkKptprdJvc7A+dWsvLnTL0D9FDVoTi1t1frUtgm\nyTb11bt9D347+W66tUoDICs/h2kfP8WagzaJXUsV7DO7kaq6sk4HFhkL/EZVv+4u3wegqo9Xs78X\nyFHV1iJyHTBRVW91t80AFqnqG1WVBXtmZ+onv7iAZ5f8jTUHnX/XPeLh5pHXclHv82st++ii6WTl\nZZOa2J5pE+8MdajVsWd2QQr2md3TIrJJRB4RkcFBlql1kmwRSQtYvJxTNcl5wMXuZNltgYvddcY0\nqISYeO49/3Yu6uUkN7/6+duK1/m/1XPw+2tuqc3Ky+bAycNk5WU3RqjmDAX1upiqThKRTjjdTWaI\nSCvgLVV9tIYyPhEpmyTbC7xcNkk2kKGqc4E73QmzfUAOcKNbNkdEHsFJmAAPq2pO/S4x9H494wsO\n5+TToV0Cj9w6LtzhmDryerzcMuo60pI78PfV/0JR3t0yn0Mns7hjzA+Ji4oNd4imAQTdz05VD6rq\ndJwBAFYDDwRR5n1V7auqvVT1MXfdA26iQ1XvU9VBqjpMVSep6uaAsi+ram/380qdr6wRHc7JZ/+R\nPA7n5Ic7FFNPIsJl/Sbzi/OmEut1uqEs37eG3yx4hpyCY2GOzjSEYEcqHiAivxGRdcAfgS9wplQ0\npkUZ3XU4D11wF23jWgOw8+hX3P/R79h9NDPMkZkzFWzN7mXgKPB1VZ2oqn9V1cMhjMuYsDm73Vk8\ndtHdnNXa6e2UXXCUBxb8npX714c5MnMmak12bivpLlV9TlWtI5KJCCkJ7Xj4wl8yIs1pjyv0FfHk\n53/hg22LwhuYqbdaGyhUtVREuolIjPsmhDERIT46jrvH38arq2fzwbZFqCovr3yL3Uf3khSTyGG3\nFfZoYS5bjuygX0qvMEdsahLs4J27gMUiMpeAybFV9ZmQRGVME+H1eLlp5DWkJXVg5up/oqos2FVx\nxJRCXxEPfPw0Px79fSb2HBumSE1tgn1mtwN4190/OeAT0Y4cK2DmuxvIOlYAwPG8YrJzC8IclQmF\nS/pO4p7xP8YjVffhVZS/ZbxOrr1j22QF28/uoVAH0txs23uUX89YQl7BqbkNThaUcMfvF/LwrePo\n3bVNGKMzodCjTTf8NbxxVOL38fme5Xyj34WNGJUJVrBdTxaKyILKn1AH11T5/crvX1tRIdGVOZFf\nwtP/WGEzWrVAwfS3y86vdhQyE2bBPrP7ZcDPccB3iOB5Y9ftOML+I9WPdJV5+CQbdmYzuFdKI0Zl\nQq1dfO219ZTEdo0QiamPYG9jK49wslhEKk+aHTEOBfGmxLyle0hLSaR96/hGiMg0hnYJbRiRNohV\nBzZUuT3aE8347uc0clQmWEElO3fk4DIenBGEW4ckomagXau4WvdZtDKTRSsz6de9LWOHpDF2SBqd\nU5MaIToTSjePvJYHFzxDdkHF21VBuPWc622qxiYs2CGeduEMjw7O7etunJfzPw9daHXTmEM8+Ur9\n3Pzoh+QcL6pTubM6JTN2SGfGDkmjZ+dWSDUte6Zpyy08zn+3LWLu5g/x+UuJi4rl1xN/Rp/2PcMR\njv0RBanGmp2InAPsVdWe7vINOM/rdgMRO8phlNfDHVeP4LFXvsRXWnEYoOgoD3deM5y8/BK+WHeA\n9Tuz8fudfyf2HDzBnoNbePOjLXRsl1Be4+t/Vjs8HvubbS5ax7Xi2iGXs+SrFRw4eZi2ca3DlehM\nHdR2GzsDmAwgIl8DHgfuAIYDLwBXhjS6Jix9QEeeuvN85izYxuK1+1GF+Fgvj/9kPL3cbiffGH82\nx/OK+XLDQZasO8CqrYcp8TnJ8VBOPm9/soO3P9lB2+RYxgxOY8yQNIb0SiE6KujBaIwxQaot2XkD\nxpG7BnhBVecAc0RkdWhDa/p6d23DPT84h1sfn8/+I3m0TY4rT3RlWiXGMHl0dyaP7k5BkY8Vmw+x\nZN0Blm88REGR06B99EQR/12ym/8u2U1ifDTnDOzIuCFpjOjXgbiYYBvMjTE1qTXZiUiUqvqAC4Gp\ndShrKomPjWL8sC6MH9aFEl8pa7YdYcm6Ayxdf4Djec5rx3kFJSxakcmiFZnERHsZ1b8DY4ekcc7A\nTiTFR4f5CoxpvmpLWG8An4jIEaAA+AxARHoDubUdXESmAM/hjFT8oqo+UWn7XcAtOI0eWcBNqrrH\n3VYKrHN3/UpVLw/2opqD6Cgv6QM6kj6gIz+5chibdmWzZN0Bvlh3gCPu62fFJaUsWXeAJesO4PUI\nQ3unMHZoZ8YM6kTbIFqEjTGn1JjsVPUxEfkYSAM+1FNNtx6cZ3fVcoeG+jMBk2SLyNzASbJxJt5O\nV9V8Efkx8Duc22WAAlUdXucraoa8HmFwrxQG90rhlisGsyMzly/W7WfJugNkHj4JQKlfWbU1i1Vb\ns/jrnDX0P6sd44amMWZwGp3aJ4b5Coxp+oIZ4mlpFeu2BnHs8kmyAUSkbJLs8mSnqgsD9l8KfC+I\n47ZoIkLvbm3o3a0NP7h0IHsPnXBqd+sPsH2v87qSKmzancOm3Tm8NHcDZ3duzdihaYwdnEb3TsnW\npcWYKoTyuVtVE12fW8P+NwP/DViOE5EMnFvcJ1T17YYPsenr1jGZbh2TuXpyXw7n5LN0vZP4Nu7M\nxu3Rws79uezcn8s/PthM55TE8i4tfbq1tS4tIZSa2L7Ct2namkQjg4h8D+etjAkBq89S1X0icjaw\nQETWqeqOSuWm4jaadO/evdHiDZcO7RK4/Gu9uPxrvcg9WcQyt0vL6q1Z5f399h/JY87C7cxZuJ32\nreMYM9hJfIPPbo/Xa11aGlIY54o19RDKZLcP6Baw3NVdV4GITAbuByaoavkrCaq6z/3eKSKLgBE4\n4+oRsM8LOP39SE9PD9swIx3aJVT4bgytk2K5+NyzuPjcs8gvLCFj0yG+WHeAFZsOUVhcCkB2biHv\nLd7Fe4t3kZwQzehBnRg3pDPD+6YSE+1tkDhsGknTXAT1uli9DiwSBWzF6bKyD2cO2O+q6oaAfUYA\ns4EpqrotYH1bIF9Vi0QkBVgCXFGpcaOCxnxdrCkrKillzdYsvli3ny83HORE/unDUMXFeBk1wOnL\nlz6gIwlx9e/SUtbHsHNKIjPum3wmoZv6secUQQpZzS7ISbKfApKAf7oP1cu6mAzAmYzbj9Py+0RN\nic6cEhvtZfSgTowe1InSUj/rd2aX9+XLzi0EoLC4lMVr9rN4zX6ivB6G901l7JA0zh3UidZJNiG0\naZlCVrNrbFazq5nfr2zbe7S8L9+BKsbj8wgMPLs9Y4c4XVo6tK39ttxqdmFnNbsgWbKLQKrKVwdP\nsGT9AZasPcDO/VX3D+/drQ1j3QaObh0rDl1UVFLKZ6symfHvdRQWl9IqMYYZ915IUkJMY1yCOcWS\nXZAs2RkOZuexdP0Bvlh7gM17cqjqT6JbxyTGDE5j3JDOtG0VywMvLOGrgycq7NMmKZaHbx1Lz84R\nO9RhOFiyC5IlO1PB0eOFLN1wkCVr97N2+xFK/af/fcREeSj2+asoDZ3aJ/D8vZPxRkD/vmnPL+Zw\nTgEd24e1Jbrl/6IbiCU7U62T+cUs3+SM0rJi82GKS0qDKjdxZFd6dW1NbEwUsdFe4mK8xMVEERvj\nJTbm1HKcuxzl9TSrtz52ZB7jjQ+3sGzDQQCivMLtVw7jwnO6h+M6ms8vLsws2ZmgFBb7WLXlMO98\ntot1O4406LE9HnEToJfY6KgKCfFUgjyVHCssR3uJdX8u2z+u0j4N2Zl6y54cfvXXxRSXnF6zvWZy\nX753yYAGO1eQLNkFqUm8QWGavriYKMYO6UzrpFju+VPDjsbv9yv5hT7yC31A3Ya6D0aUVwISYqWE\nGhvlJsyaE2pcTBSxsV7++NbqKhMdwKyPtzJ5dHcbmKGJsmRn6qTfWe3o2C6h2hnWorzCr28ag9cr\nFBWXUlRcSmGxj0L3u6i4lKKS0orLAfsUuesK3U/lYe/rw1eq+ApKqpzntyGpwqer9nH15L4hPY+p\nH0t2pk68HuHWbw3h0Ve+LJ9bI9CNlw1iZP8ODXa+0lK/kwRLTiXHwqJSikp85QmxYoI89XNRhWVf\neZItKj5VtqprOBMn8osb9Him4ViyM3V2zsBOPHbbON78aAtrtjnP76KjPPz8upGcP7xLg57L6/WQ\nGO8hMUSjNJf4/AHJr4pEGZAgc08WM2fhtiq75pSp3B/RNB2W7Ey9DO6VwqO9Upj6+HwOHMkjtU18\ngye6xhAd5SE6KoakIMdwyM4tYOGKzCq3JSfE8LVm+DuIFDbmjzkjkdYUOPVbQ+nTrc1p6+Njo7jv\nxnOIi7X6Q1Nlyc6YOkiKj+bJn57PL64fRVyMM0xWckI0z997IUN6pYQ5OlMTS3bmjHRol0DnlMRG\nHcsv3KKjPEwc2ZV27qRHyQkx5T+bpsvq3OaMRPKAneEYtNXUnyU7Y+opkhN9c2S3scaYiBDSZCci\nU0Rki4hsF5F7q9geKyJvuduXiUiPgG33ueu3iMjXQxmnMablC1myC5gk+xJgIHCdiAystNvNwFFV\n7Q08Czzplh0IXAsMAqYAf3GPZ4wx9RLKml35JNmqWgyUTZId6ArgVffn2cCF4oyRcwXwpqoWqeou\nYLt7PGOMqZdQJruqJsmu3L28fB9V9QG5QPsgyxpjTNCadQOFiEwVkQwRycjKygp3OMaYJiyUyS6Y\nSbLL93HnmW0NZAdZFlV9QVXTVTU9NTW1AUM3xrQ0oUx2y4E+ItJTRGJwGhzmVtpnLnCD+/OVwAJ1\nhk6eC1zrttb2BPoAX4YwVmNMCxfuSbJfAv4uItuBHJyEiLvfLGAj4ANuV9XgJkAwxpgq2BwUxjRv\nkTbwTL016wYKY4wJliU7Y0xEsGRnjIkIluyMMRGhxTRQiEgWsCeMIaQADTt7dPNh1x4+R1R1ShjP\n32y0mGQXbiKSoarp4Y4jHOzaI/Pamxu7jTXGRARLdsaYiGDJruG8EO4Awsiu3TR59szOGBMRrGZn\njIkIluyMMRHBkp0xJiJYsjPGRARLdsaYiGDJzhgTESzZGWMigiU7Y0xEsGTXTIlIqYisDvjcW4ey\nE0Xk3TM8/yIRqdcL8A1xfvc40SLyhIhsE5GVIrJERC450+OalilkE+6YkCtQ1eHhOLGIeMNx3io8\nAqQBg1W1SEQ6AhPCHJNpoqxm18KIyG4Redyt7WWIyEgRmSciO0TktoBdW4nIeyKyRUSeFxGPW/6v\nbrkNIvJQpeM+KSIrgasC1ntEZKaIPOouX+zWsFaKyD9FJMldP0VENrvlv90A15kA/Ai4Q1WLAFT1\nkKrOOtNjm5bJkl3zFV/pNvaagG1fubW+z4CZOHPyjgEeCthnNHAHMBDoxakEdL87PttQYIKIDA0o\nk62qI1X1TXc5CvgHsE1Vp4lICjANmKyqI4EM4C4RiQP+BnwTGAV0quqCRKRfpWsK/LSptHtv9zqP\nB/n7MhHObmObr5puY8smI18HJKnqCeCEiBQFJI0vVXUngIi8AYwHZgNXi8hUnL+NNJxkuNYt81al\n88wAZqnqY+7yGHf/xSICEAMsAfoDu1R1m3u+14CplYNW1S1AWG7NTctnya5lKnK//QE/ly2X/Tev\nPNyNikhP4JfAOap6VERmAnEB++RVKvMFMElEnlbVQpw5TD9S1esCdxKRoBKYiPTj9IRaZqKqHgtY\n3g50F5FWVrszwbDb2Mg1WkR6us/qrgE+B1rhJLRc92F/bS2bLwHvA7NEJApYCpwnIr0BRCRRRPoC\nm4EeItLLLXddVQdT1S2qOryaz7FK++a7539ORGLc86WKyFVVHdsYS3bNV+Vndk/Usfxy4E/AJmAX\n8G9VXQPKURL5AAAgAElEQVSswklOrwOLazuIqj7jlvk7kA3cCLwhImtxb2HdWt9U4D23geJwHWOt\nzjQgC9goIuuBdwGr5Zkq2eCdxpiIYDU7Y0xEsGRnjIkIluyMMRHBkp0xJiJYsjPGRIQWk+ymTJmi\nOB1l7WOfSPqYILWYZHfkyJFwh2CMacJaTLIzxpia2Luxpt78fuVAdh6+Uj9dUpOI8tq/nabpClmy\nE5GXgcuAw6o6uIrtAjwHXArkAzeq6kp32w04rwIBPKqqr4YqTlM/n67K5B8fbGb/EWdsgDbJsXxr\nQm++NbEX7ognxjQpofyneCYwpYbtlwB93M9U4K8AItIOeBA4F2fMtQdFpG0I4zR1NP/Lr3jqtRXl\niQ7g2IkiXnl3A6++tzGMkRlTvZDV7FT1UxHpUcMuVwD/p87LuUtFpI2IpAETcYYJygEQkY9wkuYb\noYq1vg7l5PPu5ztZvTULgOF9U/nm+LPp0C4hzJGFTonPz8z3NlS7/d+f7OCb559N+9bxjRiVMbUL\n5zO7LsDegOVMd11165uULXtyeOCFJeQX+srX7T5wnA+X7eGRW8fRt3vjVkZ9pX6KS0opKi6lqMT5\nlC0Xl/gpKvFRVOJ3lytvP1XmtO0lpRSV+CkuO26xD38NHR78fmXp+oN847yejXfxxgShWTdQuCPq\nTgXo3r17o5231K88/Y+VFRJdmfxCH8+8voK/3H0hqlpNEvFXk5QqLp+23eenqNhNWpXKldaUgRpZ\nfkFxuEMw5jThTHb7gG4By13ddftwbmUD1y+q6gCq+gLwAkB6enqj/d++fvsRDmRXHrT3lH1ZeXzr\nnnfwN6EEVBdejxAb4yUm2ktstPsd4/wsAut3ZNdY/p3Pd9GpfRLjh3e2xgrTZIQz2c0Ffioib+I0\nRuSq6gERmQf8NqBR4mLgvnAFWZWDOfm17tPQic7jEWLdpFOWhGKjPcTGRBET5am03ltlsjp9u+e0\nZBYT7a21C8nvX1vBJ6syq91+9EQRv3stg/982pabLh/EwJ7tG/R3YUx9hLLryRs4NbQUEcnEaWGN\nBlDV53GG874UZy6BfOCH7rYcEXkEZyRdgIfLGiuaivat42rdp1eX1qS0ia828VT82UNsdJSTfKpJ\nWk2pD9tPrhzK0ROFrN1e8a2VtJQEzurUiqXrDwKw5auj3POnzxk3NI0bvjGQzilJ4QjXGKAFjVSc\nnp6uGRkZjXIuX6mfWx77iOzcwiq3p7aN52/3TcbbhBJUQ1NV1m4/QsamQ/hK/QzulcK5gzoR5fWw\nbe9RXpq7gQ07T93uRnmFS8f15JqL+tEqMSaMkbc49pwgSJbs6mnVlsM88vIySnz+Cuujozw8eMsY\nhvVJbbRYmiJV5csNB3nl3Q3syzr1fDMxLoqrJ/fjsvE9iYn2hjHCFsOSXZAs2Z2BXftz+dei7Xy6\nch9+VeJjo/jdHefTI61Vo8bRlPlK/cxbspvXP9zC8bxTrbQd2iVww6UDOH94F2vEODP2ywuSJbsG\ncOvj89l/JI/OKYnMuG9yWGJo6vIKSpi9YBv/+XRHhdpw3+5tuOmbgxl0tjVi1JMluyC13IdKjahD\nuwQ6pyS26DcnzlRifDQ3fGMgz99zIRNHdS1fv/WrY9z758/57cwv2Z91MowRmpbOanYmLLbvPcZL\n76yv0GfP6xEuPa8n10zuS+uk2DBG16xYzS5IluxM2Kgqyzce4uV3NrAvoFbnNGL05bLxZ1sjRu0s\n2QXJkp0JO1+pn3lL9/D6vM3WiFF39osJkiU702TkFZQwZ+E2/vPJDoqtESNYluyCZMnONDmHj+bz\n2n83sXBFxVfSxg5x3sTokmpvYgSwZBckS3amydqeeYyX525g3Y5Tr6V5PcIl43pw7UX9rBHDYcku\nSJbsTJOmqizfdIhX3tlA5uFTjRgJcVFcfWFfvnl+xDdiWLILkiU70yyUlvqZt8xpxMg9GdCI0Tae\nH1w6kPOHd8Hjicj/7yPyouvDkp1pVvIL3TcxKjVi9OnWhpu+OYjBvVIaLZZfz/iCwzn5dGiXwCO3\njmu081ZiyS5I9gaFaVYS4qL5waUDef7eyVyQ3o2yHinb9h7jvr8s5rFXllXosxdKh3Py2X8kj8NB\njG9ows+SnWmWUtvG8/PrRvLM/0xgaO9Ttbml6w9y++8WMONfa8k9WRTGCE1TY8nONGu9u7bh0dvG\n8cDN59Kto9MlpdSvvLt4F1Mfn8/sBdsoLikNc5SmKQhpshORKSKyRUS2i8i9VWx/VkRWu5+tInIs\nYFtpwLa5oYzTNG8iwjkDO/HHX0ziJ1cOo43bJSW/0Mer723ktic/ZtGKvc12ThDTMEI5LLsX+DNw\nEc50iMtFZK6qls+irKo/D9j/DmBEwCEKVHV4qOIzLY/X6+GSsT2YMKILcxZu5+1F2yn2+ck6WsDT\nr6/kP5/t5KZvDmJIIzZimKYjlDW70cB2Vd2pqsXAmzgTY1fnOprgRNim+UmIi+b7lwxgxn0VGzG2\n7z3Gr/6ymEdfXkbm4RPhDdI0ulAmu6AnuxaRs4CewIKA1XEikiEiS0Xk/4UuTNNSpbRxGjGerdSI\nsWzDQW5/aiHPWyNGRGkqDRTXArNVNfBJ8lmqmg58F/iDiPSqXEhEproJMSMrK6uxYjXNTC+3EePB\nW8bQrWMy4Ex1+Z7biPHPj7dSZI0YLV4ok111k2BX5Voq3cKq6j73eyfOJNkjKhdS1RdUNV1V01NT\nI3uCG1MzESF9QEf++IuJ3F6pEeP/3t/EbU98zEJrxGjRQpnslgN9RKSniMTgJLTTWlVFpD/QFlgS\nsK6tiMS6P6cA5wEbK5c1pq68Xg9TxvZgxn0Xcs3kvuXv1R45VsAzr6/kF899wrpK8+GaliFkyU5V\nfcBPgXnAJmCWqm4QkYdF5PKAXa8F3tSK760NADJEZA2wEHgisBXXmDOVEBfN9y4ZwIx7L+TCcwIa\nMTJz+dVfnUaMvYesEaMlsXdjjQF27svl5XfWs2bbqVqdxyNMGXMW113cnzbJpw8n1URmlbN3Y4PU\nVBoojAmrs7u05pFbT2/EeP+L3VU2Yqgqpe7zvZZRXWj5aqzZicgJavhvqapNZjZoq9mZhlJa6mf+\n8q947YPNHDtxqmtKSpt4vn/JAFolxfD39zexc18uAB6B677en6su7Iu38YeZsppdkIK6jRWRR4AD\nwN9xfrnXA2mq+kBowwueJTvT0PILS/jXou38e9GOoN6vvfjcs7jj6kZ/6ceSXZCCTXZrVHVYbevC\nyZKdCZXs3AL+8cFm5i//itr+d/nTLydxVlqj3vBYsgtSsM/s8kTkehHxiohHRK4H8kIZmDFNRfvW\n8dx5zQjuu+GcWvddvHZ/I0Rk6iPYZPdd4GrgkPu5yl1nTMRo3zq+1n0KinyNEImpj6BGPVHV3dT8\nEr8xLV7XDknERntrfLWsd9c2jRiRqYuganYi0ldEPhaR9e7yUBGZFtrQjGlaEuKiuejc7tVuT2kT\nz7ihaY0YkamLYG9j/wbcB5QAqOpanDcfjIkoP7xsEKMHdjptffvWcTx4yxiioyJ6WscmLdhkl6Cq\nX1ZaZw8nTMSJifYy7abRPP6T80iMjwagTVIsz997IT0atxXW1FGwye6IO8SSAojIlTj97oyJOCLC\n4F4ptE6MAZwJu+NiQjbot2kgwf4Xuh14AegvIvuAXTgdi40xplkINtntUdXJIpIIeFTVhoMwpglY\nsWJFh6ioqBeBwdi77mX8wHqfz3fLqFGjDpetDDbZ7RKRD4C3qDh0ujEmjKKiol7s1KnTgNTU1KMe\nj8fGJAD8fr9kZWUNPHjw4ItA+XBywf5L0B+Yj3M7u0tE/iQi40MQpzGmbganpqYet0R3isfj0dTU\n1Fyc2m65YDsV5wOzgFki0hZ4DvgEsHZ24NFF08nKyyY1sT3TJt4Z7nBMZPFYojud+zupUJkLuglJ\nRCYA1wBTgAyc18cMkJWXzYGTh2vf0RgTNsG+QbEb+B/gM2CIql6tqnOCKDdFRLaIyHYRubeK7TeK\nSJaIrHY/twRsu0FEtrmfG4K/JGNMdXYd3Rv3+tq3O7266p+dl+5d2cqv/jM+5qOPPtrh7LPPHnT5\n5Zf3bIAQT3PXXXd1fuCBBzqe6XGCrdkNVdXjdTmwiHiBPwMX4cwZu1xE5lYxl8RbqvrTSmXbAQ8C\n6Th9+1a4ZY/WJQZjjMPnL2X6kpd6LM1c1b5s3XtbF5CW3KHg3vNv356W3KG4vsd+6aWXUufPn7+1\nV69eJQ0TbWjUmOxE5G5V/R3wmIic9lxAVWt6QDUa2O5OhYiIvIkzmEAwE+d8HfhIVXPcsh/h3D6/\nUWMpY0yVZq6a1SUw0ZU5cOJw/OOf/qn3s5c8uNHrqfsj+O9+97vdMzMzYy+55JI+3/rWt3J27twZ\nt3nz5nifzyf333///u9973vHpk+f3n7u3Llt8vPzPXv27Im7/fbbDxYXF3veeuut9jExMf4PP/xw\nW8eOHUuffvrplFdeeSW1pKREevToUTR79uxdycnJFaqeGzZsiL3tttu65+TkRMXFxflffPHFPSNG\njCgMJtbabmM3ud8ZwIoqPjXpAuwNWM5011X2HRFZKyKzRaRsntlgy5owe3TRdH723oM8umh6uEMx\n1cgvLvB8smtph+q2HzyZFb80c1Xr+hz79ddf/6pDhw4ln3zyyda8vDzvpEmTjq9bt27TZ599tmXa\ntGldjx8/7gHYunVr/Hvvvbdj+fLlmx5//PEuCQkJ/k2bNm1MT0/PmzFjRnuA66+//uj69es3bdmy\nZWO/fv0Kpk+fnlL5fLfccstZf/nLX77asGHDpqeeeirzxz/+cfUjM1RSY81OVd9xf1ynqivr8DsI\n1jvAG6paJCK3Aq8CFwRbWESmAlMBuncP+ppNA7LGmaZvx9E98UWlxTVWbDZlbUs6r3t67pmcZ9Gi\nRa3mzZvXZvr06Z0AioqKZPv27TEA48aNO9G2bVt/27Zt/UlJSaVXXXXVMYAhQ4bkr127NgFgxYoV\n8Q888ECXEydOePPy8rwTJkyoEE9ubq5n1apVSVdddVWvsnXFxcVBj9Qc7DO7p0WkEzAb5xnb+iDK\n7AO6BSx3ddeVU9XsgMUXgd8FlJ1YqeyiyidQ1RdwXmMjPT3dmt+NqUKMN7rWVohoT9QZ//+jqsye\nPXv7sGHDigLXf/7554kxMTHlx/d4PMTFxWnZzz6fTwCmTp3ac/bs2dvHjh1bMH369PaffPJJcuBx\nSktLSU5O9m3evLlec0gH1RqrqpOASUAWMENE1gUxnt1yoI+I9BSRGJwhoeYG7iAigYN/Xc6p2+Z5\nwMUi0tbt13exu84YU0e92/UoaBPXqsYGiHO7jjh2pueZNGnS8aeffrqj3+/k1sWLF9c+tHOA/Px8\nT/fu3UuKiorkzTffbFd5e7t27fxdu3Ytfvnll9sC+P1+lixZEvQ5gn6XTlUPqup04DZgNVDjzGKq\n6gN+ipOkNgGzVHWDiDwsImWvcNwpIhtEZA1wJ3CjWzYHeAQnYS4HHi5rrDCmqejQLoHOKYl0aJcQ\n7lBq5PV4+c6gS/dVt314p0FH+6f2zj/T8zzxxBP7fT6f9O/ff2Dv3r0HTZs2rU7P2e+99979o0eP\nHpCent6/T58+VTY6vPHGGztfeeWVlH79+g3s06fPoDlz5gQ9NHSws4sNwOlQ/B0gG+cd2Tmq2mQe\n1oRrdrFiXzE/e/83ZBccpVNSKtO/8XCjxxBOP3vvQQ6cPExaUgee+8ZD4Q4n4qxZs2bPsGHDjgSz\n7wfbFrWfs/H9LrmFJ6IBojxRel739KwfpV+XGeONaXGPgdasWZMybNiwHmXLwT6zexl4E/i6qtr0\nSTjPJ97eNI+5Wz4ir9j5R/HQySO8vWkeV/S/GBGb4c40LVP6TMye3Ov87M1Z2xOLSos9fdr1yG8V\nl1z7hLgtRK3Jzu0cvEtVn2uEeJqNt9a/w782/rfCOkV5fe3bFJeWcPXgy8IUmTHVi/J4GdyxX0RO\ng1rrMztVLQW6uY0MBjhedJK5mz+sdvt/Nn3IyaKI/HsypskKejw7YLGIzCVgcmxVfSYkUTVxaw9u\nxOevvvZf4i9h7aHNjOs+qhGjMsbUJNhkt8P9eIDkWvZt8UpKa59r6O+r57D/xEHGdB1J19Y2vZ4x\n4RbseHbWzBagb8rZte6TXXCUWevfZdb6d+nSqhPndh3BmK4jOatNF2u8MCYMgkp2IrIQd2axQKoa\n9KtdLUmXVp1I7zyUjP1rq9zeJq4VJ4vz8fmdGuC+4wf518b/8q+N/6VTUqqT+LqN5Oy23S3xmUa1\nc19u3Ger97Up8fk9A3q2Ozl2cNpxjyd8f4Pvvvtu8tNPP91x4cKF20N9rmBvY38Z8HMcTn+7iJ43\n9vZzb+CZL15g3aEtFdYP7TiAu8b9CARW7l/H0sxVrDqwgZJSZ/Sbgyez+M/mD/nP5g9JTWhXnvh6\nt++BR2y+FBMavlI/v39tRY/Fa/eXj3zyn0930CU1qeCBm8/d3jk1qd5DPDUXwd7GVh7hZLGIVJ40\nO6IkxiQwbcLP2Jq9kyc/+ysni/NoH9+W+yfcUV5bG3/WaMafNZrCkkJWHdzAsr2rWHFgPUU+59XB\nrPwc3t36Me9u/Zh28W0Y3XU4Y7qOpH9KLzweS3ym4fzt7XVdAhNdmX1ZJ+MfenFp77/cfcFGr7d+\nf3NbtmyJmTJlSp+RI0fmrVixImno0KF5N91005GHH364S3Z2dtTMmTN3Avz85z/vXlRU5ImLi/PP\nnDlzV+V3aI8fP+65+eabu1ceIqpeQVUh2NvYwPfUPDiDatZrSJiWRETol9KL5JhEThbnEeONrvK2\nNC46jrHdRjG22yiKfcWsPriRZZmryNi/loIS562YnIJjfLBtER9sW0TruFaM7jKMMd1GMjC1D/UZ\nZ8yEXnOZeySvoMTzccbeaod42n8kL37x2v2tvzaia71HPdm7d2/cW2+9tXPUqFG7hw4dOuAf//hH\n+4yMjM2vv/56m8ceeyxt1qxZu5YvX745Ojqat99+O/nuu+/uOm/evB2Bx/jVr36VNmnSpOP//Oc/\ndx85csSbnp4+4PLLLz/eqlWrMx9OmeBvY1dw6pmdD9gN3NwQAUSamKgYRncdzuiuwykpLWHdoc0s\nzVzF8n1ryt/EyC08zkc7PuOjHZ+RHJPIOW7iG9yhH1Fem3m+qWguw1ttzzwWX1RcWmO1bcPO7KQz\nSXZdunQpGj16dAFA3759Cy644ILjHo+HkSNH5j/66KOdc3JyvNdcc03P3bt3x4mIlpSUnFYrqG6I\nqJEjRwY1OGdtahup+Bxgr6r2dJdvwHlet5vgRhw2NYj2RjOy8xBGdh6Cz1/KhsNbWLZ3FV/uW83x\nopMAnCjOY8GuL1iw6wsSo+MZ1WUoY7qOZGinAcR4o8N8BaY5iIny1D7EU5T3jN6NrW4IJ6/XS2lp\nqdxzzz1dJkyYcOKjjz7asWXLlpgLLrigX+VjVDdEVEOprZowA5gMICJfAx4H7gCG44wjd2UogopE\nUR4vwzoNZFingdw86lo2ZW1nWeYqlmWu4lihM/1HXkkBn+5exqe7lxEfFcfIzoMZ020kwzsNIjbK\nXnAxVevbvW1B2+TY4qMniqr9Ixk3NK3Bno1V5fjx496uXbsWA8yYMeO0EYjh1BBRM2fO/Mrj8bB4\n8eL48847r6ChYqgt2XkDhla6BnjBnVVsjoisbqggTEVe5/1FBnfsxw9HXs3WI7tYmrmSZXtXkV3g\nzDlU4Ctk8VcZLP4qg1hvDCPSBjOm2whGpA0mPjouzFdgmhKv18O1F/Xb99d/ra1y9q9R/TscHdiz\n/RkP8VSTe+655+Att9zS88knn+x80UUXVZlYn3jiif1Tp07t3r9//4F+v1+6detW1JBdUmpNdiIS\n5Y5NdyHuEOhBljUNwCMe+qf2on9qL34w/DvsyNnDssxVLN27ksN5zkDPRaXFLM1cydLMlUR7ohiW\nNogxXUeQ3nkoCTF1Gj/RtFCXntczx68qb83f2uXYiSJniCevR782okvWT64clnkmx+7Xr1/xtm3b\nNpQtz5kzZ3dV23bv3l0+wvn06dP3A1x22WUnLrvsshMASUlJ+vrrr+85k1hqUlvCegP4RESOAAU4\n88YiIr2BMxqv3tSdRzz0ad+TPu17cv3Qb7H7WCZL9zpJ7sAJ50F5id9Hxr41ZOxbg9fjZWjHAU7i\n6zKU5NikMF+BCafLxp+dPWVsj+yNO7MTi0pKPX27t81vnRRrQzwBqOpjIvIxkAZ8qKdG+vTgPLur\nkYhMAZ4DvMCLqvpEpe13AbfgtPBmATep6h53Wymwzt31K1W9HFNOROjZths923bj2iGXszd3P0sz\nV7Fs70r2Hj8AQKm/lFUH1rPqwHo8GR4Gd+jHmG4jOKfLMFrHtQrzFZhwiPJ6GNonNSKH5Kn1VlRV\nl1axbmtt5YKcJHsVkK6q+SLyY5wJd65xtxWo6vAgriHiiQjd23She5suXD34MvYdP1h+q7v7mHOH\n4lc/aw9tYu2hTfxtxRsMTO3DuV1HMLrrcNrFBz2ytQFy8o/xzpb5HM5zBgjOKTjGxsNbGdihbzjC\n8fv9fvF4PC1upOEz4ff7BajQCh3K5261TpKtqgsD9l8KfC+E8USMLq068e2Bl/DtgZdw8GQWy/au\nYmnmSnbkOI9DVJUNh7ey4fBWXlk5i74pZzOm6wjO7TqClMTT5jkxAQ6cOMwDC35PbuGJ8nVFpcX8\nZuGzTE2/nsm9xjd2SOuzsrIGpqam5lrCc/j9fsnKymoNVJgFMZTJrqqJrs+tYf+bgcChf+NEJAPn\nFvcJVX274UNs+TolpXLFgIu5YsDFZOVlsyxzNcv2rmRL9k7AGV15y5EdbDmyg1dXz6Z3ux6M6eYk\nvo5JqWGOvul5acWbFRJdoJdXvkl6l6G0acRHBD6f75aDBw++ePDgwcHUYQKtFs4PrPf5fLcErmwS\nLaoi8j2cV9AmBKw+S1X3icjZwAIRWaeqOyqVaxKTZKcmtq/w3VSlJrbnsn4Xclm/C8nJP8aX+1az\ndO9KNh3ZTtnj2O05u9mes5vX1vybnm26cW63EYzpOoLOrTpVOFapv5QV+9dxrMjpA5hfUkCRr7hZ\n9ffz+UspLCkk31dIYUkhBb5CCsq/iyj0FZJfUkihu/5Y4XHWHtpU4/E+3/Mll/Wb3GjXMGrUqMM4\n05CaWgQ1u1i9DiwyFviNqn7dXb4PQFUfr7TfZOCPwITqZisTkZnAu6o6u7rzhWt2sZbgWOFxlmeu\nYWnmSjYc3opfT+9w3611Z3dMvhG0jWvNE5/9mW05uyvsk5rYnmkT7iQtudrXMM+IqlLi91FQUkCB\nr6jKBFX2c8UkVlRlmRJ/ww/cc1nfC/nBiEbta29jhAUplMkuCtiK0z9vH878r99V1Q0B+4wAZgNT\nVHVbwPq2QL6qFolICrAEuKJS40YFluwaxomikyzft5ZlmStZe2gzpVUMPx8XFUuhr+o3erq2SuP3\nU6aVD1elqhT5ipyEU5aEymtLRRT4CtzvwoBEVXn5VOIqrSIRNyU3jriKS/s26jCPluyCFLJkByAi\nlwJ/wOl68rLbleVhIENV54rIfGAIcMAt8pWqXi4i43BeVfPjPIf4g6q+VNO5LNk1vLzifFbsX8fS\nvStZc3Bj0DWhlIS2+FWdROUrIpR/Y8EQhLioWOKiY0mIiicuOpb4qDjio+OIj4o7bTk+OmBbVCwJ\n0afKTF/6CqsOrK/yPDHeaP76zd82dn9GS3ZBCmmya0yW7EKroKSQlQfW8cG2T9hyZEftBc6QRzzE\nR8USHx1PfFQscdFxJETHERd1KkEFLp9KUBXLxEfFERsV02ADox7Jz+HBj58mKz+nwnqPeLhjzI2c\n1/2cBjlPHViyC5IlO1MnGw9v4zcLa55ULjkmkXbxbcoTUFyFWlMscVEBiSuwNlWW1KLiiK5mbMCm\n4ETRSeZt/5R/bfwvPr+P+Kg4Hpz0c85uF5ZGsqb5S2qCmkRrrGk++qWcTbv4NuQUVD1Ihle8PH3J\nA43a/aKxJccmceWgS/ls9zIOnDxMm7hW4Up0pg6sX46pE6/Hyw+GX4lUU6H4zqBLWnSiM82X1exM\nnY3rPorYqBhmrXuHXcecfuNe8XDzqGu58OxGf4PAmKBYzc7Uy6jOQ3ji4vvokOiMw5iamMLkXuc3\n2edsxliyM/UmInjdVk5Lcaaps2RnjIkIluyMMRHBkp0xJiJYa6wx9dRcRrsxDkt2xtTTtIl3hjsE\nUwd2G2uMiQiW7IwxEcGSnTEmIliyM8ZEBEt2xpiIENJkJyJTRGSLiGwXkXur2B4rIm+525eJSI+A\nbfe567eIyNdDGacxpuULWbILmCT7EmAgcJ2IDKy0283AUVXtDTwLPOmWHQhcCwwCpgB/cY9njDH1\nEsqaXfkk2apaDJRNkh3oCuBV9+fZwIXiDJtxBfCmqhap6i5gu3s8Y4ypl1Amu6omye5S3T6q6gNy\ngfZBljXGmKA16wYKEZkqIhkikpGVlRXucIwxTVgok90+oFvAcld3XZX7uPPMtgaygyyLqr6gqumq\nmp6amtqAoZtgpSa2Jy2pg70fapq8UL4buxzoIyI9cRLVtcB3K+0zF7gBZxLsK4EFqqoiMhd4XUSe\nAToDfYAvQxirqSd7P9Q0FyFLdqrqE5GfAvM4NUn2hsBJsoGXgL+LyHYgBych4u43C9gI+IDbVfX0\nqemNMSZINm+sMc2bjYgfpGbdQGGMMcGyZGeMiQiW7IwxEaHFPLMTkSxgTxhDSAGOhPH84WTXHj5H\nVHVKGM/fbLSYZBduIpKhqunhjiMc7Noj89qbG7uNNcZEBEt2xpiIYMmu4bwQ7gDCyK7dNHn2zM4Y\nExGsZmeMiQiW7IwxEcGSnTEmIliyM8ZEBEt2xpiIYMnOGBMRLNkZYyKCJTtjTESwZNdMiUipiKwO\n+C5zSngAABrfSURBVNxbh7ITReTdMzz/IhGp1wvwDXF+9zgxIvIHEdnuft4Vke5nelzTMoVywh0T\nWgWqOjwcJxYRbzjOW4XfAslAP1UtFZEfAv8RkVGq6g9zbKaJsZpdCyMiu0Xkcbe2lyEiI0Vknojs\nEJHbAnZtJSLvicgWEXleRDxu+b+65TaIyEOVjvukiKwErgpY7xGRmSLyqLt8sYgsEZGVIvJPEUly\n108Rkc1u+W83wHUmAD8Efl42GZOqvgKcBCaf6fFNy2PJrvmKr3Qbe03Atq/cWt9nwEycaSrHAA8F\n7DMauAMYCPTiVAK63x2fbSgwQUSGBpTJVtWRqvqmuxwF/APYpqrTRCQFmAZMVtWRQAZwl4jEAX8D\nvgmMAjpVdUEi0q/SNQV+2lTavbd7nccrrc9wr8mYCuw2tvmq6TZ2rvu9DkhS1RPACREpCkgaX6rq\nTgAReQMYD8wGrhaRqTh/G2k4iWOtW+atSueZAcxS1cfc5THu/otFBCAGZ07g/sAuVd3mnu81YGrl\noFV1CxCWW3PT8lmya5mK3G9/wM9ly2X/zSsPd6PuhOa/BM5R1aMiMhOIC9gnr1KZL4BJIvK0qhbi\nTOv3kapeF7iTiASVwESkH6cn1DITVfVYwPIOoLuIJLvJvMwoYE4w5zORxW5jI9doEenpPqu7Bvgc\naIWT0HJFpCNwSS3HeAl4H5glIlHAUuA8EekNICKJItIX2Az0EJFebrnrqjqYqm5R1eHVfI5V2jcP\neBV4pqzBRER+ABQCi+v6yzAtn9Xsmq94EVkdsPyBqgbd/QRYDvwJ59nXQuDfquoXkVU4yWkvQSQN\nVX1GRFoDfweuB24E/n97dx5fRXkucPz3ZCG7JIEEQVYhURAtSy7ubFpB9KJeq1JtK62t9t62dvl0\ns/VW69J67abWVqXq1fZTtVbbil5Q0aAoCBoQWQRJWEQgkI0tK0nOc/+YCZ6sZwKZc05ynu/ncz7n\nzLwzc55hec68877zvk+LSJK7ya2qusWtGv+fiNTi3EvM6EasnbkF+BXwkYikAOXA2WqDNJoO2OCd\npk8QkROBxcBDqmqjB5t2LNkZY2KC3bMzxsQES3bGmJhgyc4YExMs2RljYkKfSXazZ89WnI6y9rJX\nLL2MR30m2VVUVEQ6BGNMFOszyc4YY7piyc4YExPscbHjoKpsKPuI9fs2A3DGoFM5LfcU3BE/+jRV\nZUvlNt4v3UBTIMC4nDFMOPE04uLs99NEJ9+eoBCRx4FLgTJVHd9BuQD3A3OAWmC+qq5xy67HGRcN\n4C5VfTLU9xUUFGhRUVFPhR/S4YZq7n37YT6q2Npq/dicMfzg3K+TnpQWtljCrb6pgd+teJT3Sze0\nWj8ycyg/Pv8bZKe2HXrO+Kjv/7L2ED9/hp8AZndRfjGQ575uBB4CEJFs4DbgTJwBJm8TkSwf4zwm\nD656ol2iA9hUXsKD74bMzb3aY6ufaZfoAHYc2MWvlz+CPYJoopFv1VhVXSYiI7vY5DLgz+4IFStF\nJFNEBgPTccZEqwIQkSU4SfNpv2Ltrk8O7uH90o2dlq/Zs54/FT1FRh+8uqtvbGDZjpWdlpdU7WBz\nRQljc/LCGJUxoUXynt1JOMMItdjlrutsfdQoqdwRcpslW9/yP5AoVVy53ZKdiTq9+m6yiNzoTg5T\nVF5eHrbvTYxPDNt39UZLt7/D2tKNBGyCLxNFInlltxsYFrQ81F23G6cqG7z+jY4O4I5btgCcBgo/\nguzIhBPHkRiXSGOgscPyxLgEbp3+bVISkjss782aAk3c9eYD1DbWdbrN7kN7+cWyBxmSMYg5+TOY\nOvIskhOSOt3emHDwdTw7957dS520xl4CfBOnNfZM4AFVneI2UKwGJrmbrgEmt9zD60y4W2Of27iI\nZze82GHZ50+/jCvGddU207u9tvUtFhQ91WFZ/6QMDjYcbrUuLTGFC0afx6wx08hJGxCOEGOJtcZ6\n5NuVnTtj1XRgoIjswmlhTQRQ1Ydx5i6YA5TgdD35sltWJSJ34gwbDnBHqEQXCVeOu5iUhCT+tflV\nDtY7s/llJp/A5WNncXHejAhH568LR59PvMTz7MaXqKzdD0B6vzTm5M/kirGzKK7cwaLiQt7dtZaA\nBqhprGPh5iW8+NFrnHnSRObkz+CUgaNjoj+iiR59ZqTicF/ZtWhqbmLnwT0ADO8/hIT42OmnHQgE\n2HlwD83azLATBtMvoV+r8oqaKl4ueZPXt71NzZHaVmWjsoZxSf4FnD1skt0DPT72i+GRJTvju/qm\nBpbtWMXiLUvZfXhvq7L+yScwa8xUPjv6fPonnxChCHs1S3YeWbIzYRPQAOv2bmZxcWG7fooJcQmc\nO7yAOfkzGZU1rJMjmA5YsvPIkp2JiN2H9rK4eClvbl9JQ/ORVmVjc/K4JH8mBUPOsGdtQ7Nk55El\nOxNR1UdqKNy2gleK36C8tnU7VE7aAC7Om86MUeeQ1i81QhFGPUt2HlmyM1GhOdBM0Z51LNpSyKby\nklZlSQlJzBh5NrPzpzMkY1CEIoxaluw8smRnos62qp0sKi5kxc7VNAWaWpVNHDyeS/JncvqgU63r\nisP+EDyyZGei1oG6g7y69S2WlCxr11F56AmDuThvBlNHnklSmy4v4XLXGw9QXlNJTtoAbp1+c0Ri\nwJKdZ7HTKcz0Opkp/bl6/KVcMXYWy3cWsXjLUrYfcMaI2HWolD+tfoqn17/ABSefy6y8aQxMzQ5r\nfOU1lZRWl4X1O82xs2Rnol5ifCLTR53NtJFnsam8hEXFhby3+wNUleojNbyw+VXn6YyhE7kkfyZ5\nA0ZZFde0Y8nO9BoiwrjcPMbl5lFWU8nLxW9QuG05tY11BDTAO5+s5p1PVjM6ewSX5M/krKGTYuqJ\nFtM168RkeqXctAF8acKVPPzvv+Ark65hcEbu0bKtVR/zwMr/5Rsv3crzGxdxqP5wF0cyscJ+9kyv\nlpyYzOy86Vw0ZiprSz9kcXEhH+zdBMD++oP8bcOL/OPDxZw3Ygpz8mcwInNohCM2kWLJzvQJcRLH\npCHjmTRkPLsOlrKoeCnLdqzkSHMjjYEmlm5fwdLtKzgtN585+TOZPPh0ezojxliyM33O0P6DubHg\nWq49/TJe2/Y2rxS/SWWdMxTVxrItbCzbwqC0gczOm86Mk88hNTElwhGbcLBkZ/qs9KQ0Lh87i0tP\nuZB3d61l8ZZCPqrcBsC+mgqeXPscz254iemjzubivOmcGHTfz/Q9luxMn5cQF885wydzzvDJlFTu\nYFHxUt7ZWUSzBqhrqmdx8VJeLn6DSUPGMyd/JuNjZKLzWONrshOR2TgTYccDj6rqPW3Kfwe0DOub\nCuSqaqZb1gysd8t2qupcP2M1sWHMgJHcPODLfOEzV/BqyTKWbH2Lww3VKMrqPetZvWc9w/oPYU7e\nDM4fMaXdgKSm9/JzWPZ44A/AZ3GmQ3xPRBaq6oct26jqd4O2/xYwMegQdao6wa/4TGzLTslk3ulz\n+Y9xF/P2x++xeEshHx/cDTjzAj9S9FeeWvcvLhx9PrPGTCM7NbPdMQL0jUctY4Vvz8aKyNnA7ao6\ny12+BUBVf9nJ9iuA21R1ibtcrarpXr/Pno01x0NV2Vi2hUXFS1m9ex0alMjiJY6zhk1ijvt0RnHl\ndp5Zv5D1+zYfLZ8/8WouGjM1EtVfq2975Gc1tqPJrs/saEMRGQGMAgqDVieLSBHQBNyjqv/yK1Bj\nRITxg05h/KBT2FddzuLiN1i6bQV1TfU0a4DlO4tYvrOIoScMpvTwPpqD5sRt1gCPrXmGfTUVfGnC\nlRE8C9OVaOloNA94TlWbg9aNUNUC4FrgPhEZ3XanSE2Sbfq2Qek5zJ94FQ/P/SVfnng1g9Jzjpbt\nOlTaKtEFe+mj1yg9bAMDRCs/k11nk2B3ZB7wdPAKVd3tvm/DmSR7YtudVHWBqhaoakFOTk7bYmOO\nS0piMhfnz+D+Obfzo/P/i/wBo0Lus2Kn3UqJVn4mu/eAPBEZJSL9cBLawrYbicipQBbwTtC6LBFJ\ncj8PBM4FPmy7rzHhECdxTB5yOvMnXh1y27ZTRpro4ds9O1VtEpFvAq/gdD15XFU3isgdQJGqtiS+\necAz2rqlZCzwiIgEcBLyPcGtuMZEwokZOSTGJdDYZvTkYMMzTwpjRKY7fO1np6qLgEVt1v2szfLt\nHey3Ajjdz9iM6a70fmmcP2IKhdtXdFjePymDs4dNDnNUxqtoaaAwple4fuJVjM0Z0259WmIqPzjv\n6xEbIt6E1uWVnYgchs57TqqqTeFuYkpKYjI/m/4dVu9Zzx9WPUldUz0Z/dK5b85tZCR57hZqIqDL\nKztVzXAT2v3Aj3H6zg0FfgTc5394xkSf+Lh4pgydQGay81uf3i/VEl0v4LUaO1dV/6iqh1X1kKo+\nBFzmZ2DGGNOTvCa7GhG5TkTiRSRORK4DavwMzBhjepLXZHctcDWwz31d5a4zxphewVPXE1XdgVVb\njTG9mKcrOxHJF5HXRWSDu3yGiNzqb2jGGNNzvFZj/wTcAjQCqOo6nCcfjDGmV/Ca7FJV9d026zp/\nZsYYY6KM12RX4Q6xpAAi8jmg1LeojDGmh3l9NvYbwALgVBHZDWwHrvMtKmOM6WFek93HqnqhiKQB\ncap62M+gjDGmp3mtxm4XkQXAWUC1j/EYY4wvvCa7U4HXcKqz20XkQRE5z7+wjIl+OWkDGJyeS07a\ngEiHYjzo9uxiIpKFMzDAdaoa70tUx8BmFzMxymYX88jzeHYiMk1E/gisBpJxHh8Ltc9sEflIREpE\n5McdlM8XkXIRWeu+vhpUdr2IFLuv673GaYwxHfHUQCEiO4D3gWeBH6hqyEEAvEyS7fqbqn6zzb7Z\nwG1AAU53l9Xuvvu9xGuMMW15bY09Q1UPdfPYU4ASd3YwROQZnOdrvcwlMQtYoqpV7r5LgNm0mYHM\nGGO8CjVS8Q9V9V7gbhFpd3NPVW/uYnevk2RfKSJTgS3Ad1X1k072tZlMjDHHLNSV3Sb33a87/y8C\nT6tqg4jcBDwJzPS6s4jcCNwIMHz4cH8iNMb0CV0mO1V90f24XlXXdPPYISfJVtXKoMVHgXuD9p3e\nZt83OohvAc6THRQUFHSvWdkYE1O8tsb+RkQ2icidIjLe4z4hJ8kWkcFBi3P59EryFeAid7LsLOAi\nd50xxhwTr4N3zhCRE3G6mzwiIifgtKLe1cU+XibJvllE5uKMoFIFzHf3rRKRO3ESJsAdLY0Vxhhz\nLI6lU/HpwA+Ba1Q1aibJtE7FJkZZp2KPvI5UPFZEbheR9cDvgRU499GMMaZX8NrP7nHgGWCWqu7x\nMR5jjPFFyGTnPgmxXVXvD0M8xhjji5DVWFVtBoa5LarGGNMrea3GbgeWi8hCgibHVtXf+hKVMcb0\nMK/Jbqv7igMy/AvHGGP84bWf3c/9DsQYY/zkdYinpbgziwVTVc/PsRpjTCR5rcZ+P+hzMnAlNm+s\nMaYX8VqNXd1m1XIRaTtptjHGRC2v1djsoMU4nBGE+/sSkTHG+MBrNXY1n96zawJ2ADf4EZAxxvgh\n1EjF/wZ8oqqj3OXrce7X7cDb8OrGGBMVQj1B8QhwBMAdOv2XOKMJH8QdNNMYY3qDUNXY+KBx5K4B\nFqjq88DzIrLW39CMMabnhLqyixeRloR4AVAYVOb1fp8xxkRcqGT3NPCmiLwA1AFvAYjIGJyqbJc8\nTJL9PRH5UETWicjrIjIiqKw5aPLshW33NcaY7gg14c7dIvI6MBh4VT8d1jgO+FZX+3qcJPt9oEBV\na0XkP3Em3LnGLatT1QndPiNjjOlAyKqoqq7sYN0WD8cOOUm2qi4N2n4l8AUPxzXGmG7zOrvYseju\nRNc3AIuDlpNFpEhEVorI5X4EaIyJHVHRyCAiX8B5KmNa0OoRqrpbRE4GCkVkvapubbOfTZJtjPHE\nzyu7kJNkA4jIhcBPgbmq2tCyXlV3u+/bcCbInth2X1VdoKoFqlqQk5PTs9EbY/oUP5Odl0myJ+J0\nXJ6rqmVB67NEJMn9PBA4F3tiwxhzHHyrxnqcJPtXQDrwdxEB2Kmqc4GxOJNxB3AS8j1tWnGNMaZb\nuj1JdrSySbJNjLJJsj3ysxprjDFRIypaY43pjf77kRWUVdWSm53KnTedE+lwTAiW7Iw5RmVVteyp\nqAm9oYkKVo01xsQES3bGmJhgyc4YExMs2RljYoIlO2NMTLBkZ0w3qSrrSyo4WHMEgNr6JuobbM74\naGfJzphuaGhs5o7HVvGTh5ZTU9cIwIHqBm6653W27wk5eLeJIEt2xnTDEy9upGjTvnbrqw7Vc8dj\nq2hsao5AVMYLS3bGeHSwup5XVu3otLziQB0r1pWGLyDTLfYEhTGu+iNNlO+vo2x/LWVVtZTtr3Pf\nnVfVoYaQxyjZdYBpk4aGIVrTXZbsTMyorW9slcD2VdVSvr+OfftrKd9fy8HqI8f9HSlJ9l8qWtnf\njOkTVJXDtY2UuYlrX1Wd+/5pQmtpUOiulKR4crNSyclKZfPHVVTXdn6ccz8z5FhPwfjMkp3pFVSV\nA9UNTuKqqj2ayMrcamf5/lrqGo6tcSA9JZHcrFRys1Pc91Rys1LIyUplUHYq6SmJuIPLsmZzGT9/\nbCWBQPtxIGedNYIRJ55wXOdp/GPJ7jjsqahm4bJtrN1SDsDE/BzmTh3N4IFpEY7MfxUH6lj41jaK\nNu2luVk57eQBXDZt9DH/Zw8ElP2H6z9NYC33ytzl8v21HGkKHNOx+6f3c5JYUCJz3p3PqcmJno81\n6dRcfv61s3hy0SZKPjkAQJwI180+lStn5h1TfCY8fB2pWERmA/fjDMv+qKre06Y8CfgzMBmoBK5R\n1R1u2S040ys2Azer6itdfVe4Ryr+cHslty14h/ojra8mUpISuOPGszl1ZHbYYgm3HaWH+OlDyzlU\n0/oeV2JCHD+ZP4WCsYPa7dPcHKDiYH2nN/8rDtTR1Nz9f4sikJWR3DqBtSS0rFRyslJI7ufPb/pX\n717CvqpaBg9MY8EtF/ryHR7YSMUe+XZlJyLxwB+Az+LMGfueiCxsM5fEDcB+VR0jIvOA/wGuEZFx\nOBP0nAYMAV4TkXxVjYpOTM0B5TdPrWmX6ADqGpr47VNrePjHFxAX1/f+Haoq9//t/XaJDqCxKcC9\nfyli/qXjqGpJbG41s/JAHR3U/EKKExiY6VQpgxPaoKxUcrJTyMlMITEhvgfOrPvi3b/fvve33Df5\nWY2dApS4UyEiIs8Al9F6lrDLgNvdz88BD4pzc+Qy4Bl3asXtIlLiHu8dH+P1bF1xOWVVtZ2Wl1bW\n8LVfLCHJpyuKSGpsamZvZefnXtfQxEPPr/N8vIR4ISfz0/tlzn0y935ZVioD+icTH2/dQc3x8/N/\n40nAJ0HLu4AzO9vGnY3sIDDAXb+yzb4n+Rdq95Ttr+uRbWJBv4S4dtXLlkSWm51CVkZyn7wCNtGn\nV196iMiNwI0Aw4cPD9v35mSmhNwmKyOJfomRqV75qakpQOWh+i63ufS8k5kxeSi5Wan0T+93tCXT\nmEjyM9ntBoYFLQ9113W0zS4RSQD64zRUeNkXVV0ALACngaLHIg/hM3kDGZiZQsWBjq/ecrNTWXDL\nhUfv6fQlqsp3fvcm23Z3/NB7ekoi118y1rdGgWiSm53a6t1ENz9vhrwH5InIKBHph9PgsLDNNguB\n693PnwMK1WkeXgjME5EkERkF5AHv+hhrt8THx/G9z0+iX2L7P76kxHi+O29in0x0ACLCzVdPIC2l\nfXeN+Hjh2/MmxkSiA7jzpnN45JYLbWaxXsLvridzgPtwup48rqp3i8gdQJGqLhSRZOAvwESgCpgX\n1KDxU+ArQBPwHVVd3NV3RWKS7E/2Heafb5Swtriln10ul08bzbBBGWGNIxL2VtbwwrKtvPfhPpqb\nA4wfM5DLp45m9NDMSIcWa/rmr6oPfE124RSJZGdMFLBk55G16RtjYoIlO2NMTLBkZ4yJCX3mnp2I\nlAMfRzCEgUBFBL8/kuzcI6dCVWdH8Pt7jT6T7CJNRIpUtSDScUSCnXtsnntvY9VYY0xMsGRnjIkJ\nlux6zoJIBxBBdu4m6tk9O2NMTLArO2NMTLBkd5xE5HERKRORDZGOJdxEZJiILBWRD0Vko4h8O9Ix\nhYuIJIvIuyLygXvuP490TKZrVo09TiIyFagG/qyq4yMdTziJyGBgsKquEZEMYDVweZuh9/skd0Tt\nNFWtFpFE4G3g26q6MsSuJkLsyu44qeoynBFbYo6qlqrqGvfzYWATUTSitJ/UUe0uJrovu3KIYpbs\nTI8QkZE4Q3Wtimwk4SMi8SKyFigDlqhqzJx7b2TJzhw3EUkHnscZd/BQpOMJF1VtVtUJOCNpTxGR\nmLqN0dtYsjPHxb1f9TzwV1X9R6TjiQRVPQAsBewZ1Shmyc4cM/cm/WPAJlX9baTjCScRyRGRTPdz\nCs78yJsjG5XpiiW74yQiT+PMZ3uKiOwSkRsiHVMYnQt8EZgpImvd15xIBxUmg4GlIrIOZ76VJar6\nUoRjMl2wrifGmJhgV3bGmJhgyc4YExMs2RljYoIlO2NMTLBkZ4yJCZbszFEi0ux2H9kgIn8XkdQu\ntr1dRL4fzviMOR6W7EywOlWd4I7ecgT4eqQDMqanWLIznXkLGAMgIl8SkXXu2G1/abuhiHxNRN5z\ny59vuSIUkavcq8QPRGSZu+40dxy4te4x88J6ViZmWadic5SIVKtquogk4Dzv+jKwDPgncI6qVohI\ntqpWicjtQLWq/lpEBqhqpXuMu4B9qvp7EVkPzFbV3SKSqaoHROT3wEpV/auI9APiVbUuIidsYopd\n2ZlgKe6QRUXATpznXmcCf1fVCgBV7WjsvvEi8pab3K4DTnPXLweeEJGvAfHuuneAn4jIj4ARluhM\nuCREOgATVercIYuOcp71D+kJnBGKPxCR+cB0AFX9uoicCVwCrBaRyar6lIisctctEpGbVLWwB8/B\nmA7ZlZ0JpRC4SkQGAIhIdgfbZACl7nBP17WsFJHRqrpKVX8GlAPDRORkYJuqPgC8AJzh+xkYg13Z\nmRBUdaOI3A28KSLNwPvA/Dab/TfOCMXl7nuGu/5XbgOEAK8DHwA/Ar4oIo3AXuAXvp+EMVgDhTEm\nRlg11hgTEyzZGWNigiU7Y0xMsGRnjIkJluyMMTHBkp0xJiZYsjPGxARLdsaYmPD/q49hVguCaHQA\nAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x3c3eed0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# grid = sns.FacetGrid(train_df, col='Embarked')\n",
    "grid = sns.FacetGrid(train_df, row='Embarked', size=2.2, aspect=1.6)\n",
    "grid.map(sns.pointplot, 'Pclass', 'Survived', 'Sex', palette='deep')\n",
    "grid.add_legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 类别和数值型特征的相关性\n",
    "分析类别特征和数值型特征的相关性。包括Embarked (Categorical non-numeric), Sex (Categorical non-numeric), Fare (Numeric continuous), with Survived (Categorical numeric)\n",
    "\n",
    "### Observations\n",
    "1. fare越高存活率越高\n",
    "2. 上船口和存活率相关\n",
    "\n",
    "### Decisions\n",
    "1. 把fare划分区间加入模型中"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/lib/python2.7/site-packages/seaborn/axisgrid.py:703: UserWarning: Using the barplot function without specifying `order` is likely to produce an incorrect plot.\n",
      "  warnings.warn(warning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x7c46c10>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgAAAAHUCAYAAABMP5BeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8JHV95//XmxkJNxHEIyLIDxQCyxpFmCAsWUXgR8aN\nEZIQkFUDLmb0sd4i8RcxmgiJ9yQirsZAQCEJERBUWDQIS8AEL8AAozhcwoTr8ONyMCAXFRzms390\nDTSHnpk+lzo9Z+r1fDzO43RV17fq093n2+fd36quSlUhSZK6ZYNRFyBJkmafAUCSpA4yAEiS1EEG\nAEmSOsgAIElSBxkAJEnqIAOAJEkdZACYhiRPJFnS93PsJNrul+SCaW7/siQLpth22ttv1vO6JNcm\n+UGS65O8bcAyOyS5bDXt/0eS65L8MMmPkhw83Zqa9S5I8tkZWtdtSZ43zXUkyWeTLGse6x4zUZsm\nxz5rn53EOnZN8r0kjyV530zUta6ZP+oC5rifVdXuo9hwknmj2O6EGp4FnAzsVVXLk/wSsMMk2m8H\nfBDYo6p+kmQzYGwS7edX1YpB91XVYmDxsOuaBa8Fdm5+Xgl8ofmt2WWftc8O6z+AdwOHjLqQtjgC\n0IImfX68+YSxOMkeSb6V5N+TvL1v0c2TfCPJTUn+JskGTfsvNO2WJjl+wno/meQa4Hf75m+Q5LQk\nH2mmD2qS6zVJvtJ0UpIsTHJj0/63Z+ChPpteiPwxQFU9VlU3TaL984GHgUea9o9U1a1NrU9+Ukry\nvCS3NbePSnJ+kn8GLklyZpLfWLXC5nk4dNWnpea5uS3JFn3L3Jxk6yRjSc5NclXzs29z/1ZJLmqe\n/1OATP0petLBwN9Vz/eBLZJsMwPr1Qywzw6tM322qu6rqquAX0x3XesqA8D0bJynDyce3nffHc0n\njX8FTgMOBfYGju9bZi/gXcBuwEt4qoN/sKoWAC8DXp3kZX1tflxVe1TVmc30fOAM4Oaq+lB6w14f\nAg6sqj3oJepjkmwE/C3wm8CewAsGPaAku0x4TP0/W/QvW1X/AZwP3J7ky0neuOoNcUg/AO4Fbk3y\npSS/OWS7PYBDq+rVwFnAYU3tGwIHAN/oq3ElcB7wW80yrwRur6p7gROBE6rqV4HfAU5pmn0YuLyq\n/jPwNWD7QUUkOWs1z9PvDVh8W+DOvunlzTzNLvusfXbYPrvecxfA9KxpOPH85vd1wGZV9TDwcHr7\nk1Z1yiur6haAJF8Gfg04BzgsySJ6r8829N5sfti0OWvCdk4Czq6qjzbTezfLfycJwIbA94BdgVur\n6uZme/8ALJpYdPNpYOgh0qp6a5JfAQ4E3gf8v8BRQ7Z9IslC4FfpvQmckGTPqjpuLU0vbt7IAP4J\nODG9ocyFwL9U1c+ax77KWcCfAl8C3sBTz+GBwG59y27efPJ6Fc0be1V9I8kDq6n/8EHztU6zz9pn\n1TAAtOex5vfKvturplc97xOvxFRJdqTXKX+1qh5IchqwUd8yj05o813gNUn+qqp+Tm/o6+KqOqJ/\noSRDvUEk2YVnvmGtsl9VPThxZlVdB1yX5O+BWxnyzaRpW8CVwJVJLqbX4Y8DVvDUCNVGE5o92tf+\n5+kdrPTrwOHAmTzT94CdkozR25/3kWb+BsDezfP2pAlvRKuV5CxglwF3fbqq/m7CvLuAF/VNb9fM\n07rDPjuEDvXZ9Z67AEZrryQ7NkNwhwOXA5vT6yw/SbI1vYPH1uRU4JvA2UnmA98H9k2yE0CSTZP8\nMnAjsEOSlzTtjhi0sqq6qap2X83P095IkmyWZL++WbsDtw/74JO8ME8/Gr6//W30hj2hNxS7JmcB\nbwH+K3DhgMdU9IYFPw3cUFU/bu66iN5w7qp6Vr3h/gvw35t5rwW2HLTRqjp8Nc/ToDeS84HfS8/e\nwE+q6u61PC6te+yz3emz6z1HAKZn4yRL+qYvrKqhv1YEXAV8DtgJuBT4WlWtTHItvc5/J/Cdta2k\nqj6d5DnA3wNvpJfmv9wMsQF8qKr+rRmi/EaSn9Lbz/nsSdQ6SIA/SnIS8DN6b4JHTaL9s4C/TPJC\n4OfAOLDqgKu/pPcGuYi+/YOrcRG9x35eVT2+mmXOovd899f3buDzSX5Iry/8S7P94+k9f0vpfVq7\nYxKPaXW+Cfw3YBnwU3pvfpp99ln77FCSvIDe8RibAyuT/AGwW1U9NN11ryvSC1pSe5LsAJxWVfuN\nthJJw7DPdoO7ACRJ6iADgGbDg/S+ViVpbrDPdoC7ACRJ6iBHACRJ6qA58S2AhQsX1oUXPuObIpJm\n3kyc9tg+K82eKffZOTECcP/994+6BEmTYJ+V1n1zIgBIkqSZZQCQJKmDDACSJHWQAUCSpA5qNQAk\neW+SpUl+lN61pzdqLqRxRZJl6V2becM2a5AkSc/UWgBIsi29CzcsqKqXAvPoXdf5k8AJVbUT8ABw\ndFs1SJKkwdreBTCf3tW35gObAHcD+wPnNPefTu9az5IkaRa1FgCq6i56l4e8g94//p8AVwMPVtWK\nZrHlwLZt1SBJkgZrcxfAlsDBwI7AC4FNgYWTaL8oyeIki8fHx1uqUtJMsc9Kc0ubuwAOBG6tqvGq\n+gXwVWBfYItmlwDAdsBdgxpX1clVtaCqFoyNjbVYpqSZYJ+V5pY2A8AdwN5JNkkS4ADgeuBS4NBm\nmSOB81qsQZIkDdDmMQBX0DvY7xrgumZbJwPvB45JsgzYCji1rRokSdJgrV4NsKo+DHx4wuxbgL3a\n3K4kSVozzwQoSVIHGQAkSeogA4AkSR1kAJAkqYMMAJIkdZABQJKkDjIASJLUQQYASZI6yAAgSVIH\nGQAkSeogA4AkSR1kAJAkqYMMAJIkdZABQJKkDjIASJLUQQYASZI6yAAgSVIHGQAkSeqgVgNAki2S\nnJPkxiQ3JNknyXOTXJzk5ub3lm3WIEmSnqntEYATgQuralfg5cANwLHAJVW1M3BJMy1JkmZRawEg\nyXOAVwGnAlTV41X1IHAwcHqz2OnAIW3VIEmSBmtzBGBHYBz4UpJrk5ySZFNg66q6u1nmHmDrFmuQ\nJEkDtBkA5gN7AF+oqlcAjzJhuL+qCqhBjZMsSrI4yeLx8fEWy5Q0E+yz0tzSZgBYDiyvqiua6XPo\nBYJ7k2wD0Py+b1Djqjq5qhZU1YKxsbEWy5Q0E+yz0tzSWgCoqnuAO5Ps0sw6ALgeOB84spl3JHBe\nWzVIkqTB5re8/ncBZyTZELgFeAu90HF2kqOB24HDWq5BkiRN0GoAqKolwIIBdx3Q5nYlSdKaeSZA\nSZI6yAAgSVIHGQAkSeogA4AkSR1kAJAkqYMMAJIkdZABQJKkDhoqAKTnTUn+tJnePsle7ZYmSZLa\nMuwIwF8D+wBHNNMPA59vpSJJktS6Yc8E+Mqq2iPJtQBV9UBzel9JkjQHDTsC8Isk82gu3ZtkDFjZ\nWlWSJKlVwwaAzwJfA56f5KPA5cDHWqtKkiS1aqhdAFV1RpKr6V3EJ8AhVXVDq5VJkqTWrDUANEP/\nS6tqV+DG9kuSJEltW+sugKp6ArgpyfazUI8kSZoFw34LYEtgaZIrgUdXzayq17dSlaTOOOmiJaMu\nYb32toN2H3UJWkcNGwD+pNUqJEnSrBr2IMBvT3UDzTEEi4G7qup1SXYEzgS2Aq4G3lxVj091/ZIk\nafKGPRXw3kmuSvJIkseTPJHkoSG38R6g/xsDnwROqKqdgAeAoydXsiRJmq5hzwPwOXqnAb4Z2Bh4\nK0OcCjjJdsBvAKc00wH2B85pFjkdOGRyJUuSpOka+mqAVbUMmFdVT1TVl4CFQzT7DPBHPHXWwK2A\nB6tqRTO9HNh2EvVKkqQZMGwA+Glz7v8lST6V5L1ra5vkdcB9VXX1VApLsijJ4iSLx8fHp7IKSbPI\nPivNLcMGgDc3y76T3tcAXwT8zlra7Au8Pslt9A762x84EdgiyaqDD7cD7hrUuKpOrqoFVbVgbGxs\nyDIljYp9Vppb1vYpfnuAqrq9qn5eVQ9V1fFVdUyzS2C1quoDVbVdVe0AvAH456p6I3ApcGiz2JHA\nedN+FJIkaVLWNgLw9VU3kpw7Q9t8P3BMkmX0jgk4dYbWK0mShrS28wCk7/aLp7qRqroMuKy5fQuw\n11TXJUmSpm9tIwC1mtuSJGkOW9sIwMubE/4E2Ljv5D8Bqqo2b7U6SZLUijUGgKqaN1uFSJKk2TP0\niYAkSdL6wwAgSVIHGQAkSeogA4AkSR1kAJAkqYPW9jVASZKe4aSLloy6hPXW2w7afVa24wiAJEkd\nZACQJKmDDACSJHWQAUCSpA4yAEiS1EF+C0Aj59HE7ZqtI4olzS2OAEiS1EEGAEmSOqi1AJDkRUku\nTXJ9kqVJ3tPMf26Si5Pc3Pzesq0aJEnSYG2OAKwA/rCqdgP2Bt6RZDfgWOCSqtoZuKSZliRJs6i1\nAFBVd1fVNc3th4EbgG2Bg4HTm8VOBw5pqwZJkjTYrBwDkGQH4BXAFcDWVXV3c9c9wNazUYMkSXpK\n6wEgyWbAucAfVNVD/fdVVQG1mnaLkixOsnh8fLztMiVNk31WmltaPQ9AkmfR++d/RlV9tZl9b5Jt\nquruJNsA9w1qW1UnAycDLFiwYGBIGMTvlLfH75NrTabaZyWNRpvfAghwKnBDVX26767zgSOb20cC\n57VVgyRJGqzNEYB9gTcD1yVZ9bH8j4FPAGcnORq4HTisxRokSdIArQWAqrocyGruPqCt7UqSpLXz\nTICSJHWQAUCSpA4yAEiS1EEGAEmSOsgAIElSBxkAJEnqIAOAJEkdZACQJKmDDACSJHWQAUCSpA4y\nAEiS1EEGAEmSOsgAIElSBxkAJEnqIAOAJEkdZACQJKmDDACSJHWQAUCSpA4aSQBIsjDJTUmWJTl2\nFDVIktRlsx4AkswDPg+8FtgNOCLJbrNdhyRJXTaKEYC9gGVVdUtVPQ6cCRw8gjokSeqsUQSAbYE7\n+6aXN/MkSdIsmT/qAlYnySJgUTP5SJKbRllPi54H3D/qIobx9lEXsO6YM68ZTPp1u7CqFk5lO/bZ\ndZP99klz5nWbtT5bVVNpN2VJ9gGOq6pfb6Y/AFBVH5/VQtYRSRZX1YJR16Hh+Zp1m6//3OTr9kyj\n2AVwFbBzkh2TbAi8ATh/BHVIktRZs74LoKpWJHkn8C1gHvDFqlo623VIktRlIzkGoKq+CXxzFNte\nB5086gI0ab5m3ebrPzf5uk0w68cASJKk0fNUwJIkdZABQJKkDjIASJLUQQYASZI6yAAgSVIHGQAk\nSeogA4AkSR1kAJAkqYMMAJIkdZABQJKkDjIASJLUQQaAKUryRJIlfT/HTqLtfkkumOb2L0sypWtb\nz8T2m/U8K8knktyc5Jok30vy2gHLHZfkqAHzN0lyRpLrkvwoyeVJNptuXc26/yzJgTOwnpl6rnZM\nckWSZUnOai6FrVlmv7XfTnI972z6bCV53nTXt64ZydUA1xM/q6rdR7HhJPNGsd0B/hzYBnhpVT2W\nZGvg1ZNo/x7g3qr6FYAkuwC/GLZxkvlVtWLQfVX1p5OoYzZ8Ejihqs5M8jfA0cAXRlxTF9lv7beT\n8R3gAuCyEdfRCkcAZliS25J8vPl0sTjJHkm+leTfk7y9b9HNk3wjyU1J/ibJBk37LzTtliY5fsJ6\nP5nkGuB3++ZvkOS0JB9ppg9qEv01Sb6yKpknWZjkxqb9b8/A49wE+H3gXVX1GEBV3VtVZ09iNdsA\nd62aqKqbmjekHZL8qG9b70tyXHP7siSfSbIY+GCS2/ueu02T3Nl8wjktyaHN4/5K37qe/GQwi89V\ngP2Bc5pZpwOHTHe9mjn2W/vtIFV1bVXdNhPrWhcZAKZu4zx9KPHwvvvuaD5l/CtwGnAosDdwfN8y\newHvAnYDXsJTf7AfrKoFwMuAVyd5WV+bH1fVHlV1ZjM9HzgDuLmqPpTeENWHgAOrag9gMXBMko2A\nvwV+E9gTeMGgB5RklwmPqf9niwmL79Q8zoeGfL4G+SLw/qYzfyTJzkO227CqFlTV8cASnvr08jrg\nW1XV/2nk/wCvTLJpM304cOYsP1dbAQ/2fepZDmw75GPVzLLf2m+Hfa7We+4CmLo1DSWe3/y+Dtis\nqh4GHk7yWN8f2ZVVdQtAki8Dv0bvE+JhSRbRe222ofdG88OmzVkTtnMScHZVfbSZ3rtZ/ju9D51s\nCHwP2BW4tapubrb3D8CiiUVX1U3ArA2PVtWSJC8GDgIOBK5Ksg/ws7U0PWvC7cOBS4E3AH89YRsr\nklwI/GaSc4DfAP6I3pvPnHmuNGPst9Nkv11/GADa8Vjze2Xf7VXTq57zmtCmkuwIvA/41ap6IMlp\nwEZ9yzw6oc13gdck+auq+jkQ4OKqOqJ/oSRD/cGnty9v4pvVKvtV1YN908uA7ZNsPp1PE1X1CPBV\n4KtJVgL/ramhf3RqownN+p+H84GPJXkuveT/zwM2cybwTuA/gMVV9XB67x6z9Vz9GNgiT+373I6+\nIVStM+y3Q+pIv13vuQtgdPZK78jwDegl4cuBzel1kp+kd2DOM47MneBU4JvA2UnmA98H9k2yEzy5\nb+2XgRuBHZK8pGl3xKCVNfvydl/Nz4MTlv1ps/0T0xzRnmQsye8OWvcgSfZNsmVze0N6yf524F7g\n+Um2SvJL9IYIB2reiK4CTgQuqKonBiz2bWAPevs+Vw3DzuZzVfQ+6RzazDoSOG8NT43WXfbbjvTb\nLjAATN3EfYmfmGT7q4DPATcAtwJfq6ofANfS+2P+R3pHoK5RVX26afP39D5pHgV8OckPaYbGmk8Z\ni4BvpHeAzH2TrHV1PgSMA9end/DPBcBkPlW8BPh2kuuax7AYOLfZF/hnwJXAxfSejzU5C3gTq0n2\nzZvLBfTemC9o5o0zu8/V++ntq1xG75iAU2dovZoc+639dmhJ3p1kOb1Rux8mOWUm1ruuSO/DidSe\n9I4Evq2qThtxKZKGZL9d/zkCIElSB3kQoGbDZUDn9q9Jc9xl2G/Xa+4CkCSpg9wFIElSBxkAJEnq\noDlxDMDChQvrwgsvHHUZUhdkJlZin5VmzZT77JwYAbj//vtHXYKkSbDPSuu+OREAJEnSzDIASJLU\nQQYASZI6aE4cBChp/XXSRUtGXcJ67W0HefVbDdbqCECS9yZZmuRHSb6cZKPmSlpXJFmW5KxVV6SS\nJEmzp7UAkGRb4N3Agqp6KTAPeAPwSeCEqtoJeAA4uq0aJEnSYG0fAzCf3uU35wObAHcD+wPnNPef\nDhzScg2SJGmC1gJAVd0F/CVwB71//D8BrgYerKoVzWLLgW3bqkGSJA3W5i6ALYGDgR2BFwKbAgsn\n0X5RksVJFo+Pj7dUpaSZYp+V5pY2dwEcCNxaVeNV9Qvgq8C+wBbNLgGA7YC7BjWuqpOrakFVLRgb\nG2uxTEkzwT4rzS1tBoA7gL2TbJIkwAHA9cClwKHNMkcC57VYgyRJGqDNYwCuoHew3zXAdc22Tgbe\nDxyTZBmwFXBqWzVIkqTBWj0RUFV9GPjwhNm3AHu1uV1JkrRmngpYkqQOMgBIktRBBgBJkjrIACBJ\nUgcZACRJ6iADgCRJHdTq1wAlSeunky5aMuoS1ltvO2j3WdmOIwCSJHWQAUCSpA4yAEiS1EEGAEmS\nOsgAIElSBxkAJEnqIAOAJEkdZACQJKmDDACSJHWQAUCSpA5qNQAk2SLJOUluTHJDkn2SPDfJxUlu\nbn5v2WYNkiTpmdoeATgRuLCqdgVeDtwAHAtcUlU7A5c005IkaRa1FgCSPAd4FXAqQFU9XlUPAgcD\npzeLnQ4c0lYNkiRpsDZHAHYExoEvJbk2ySlJNgW2rqq7m2XuAbZusQZJkjRAmwFgPrAH8IWqegXw\nKBOG+6uqgBrUOMmiJIuTLB4fH2+xTEkzwT4rzS1tBoDlwPKquqKZPodeILg3yTYAze/7BjWuqpOr\nakFVLRgbG2uxTEkzwT4rzS2tBYCquge4M8kuzawDgOuB84Ejm3lHAue1VYMkSRpsfsvrfxdwRpIN\ngVuAt9ALHWcnORq4HTis5RokSdIErQaAqloCLBhw1wFtbleSJK2ZZwKUJKmDDACSJHWQAUCSpA4y\nAEiS1EEGAEmSOsgAIElSBxkAJEnqIAOAJEkdNFQASM+bkvxpM719kr3aLU2SJLVl2BGAvwb2AY5o\nph8GPt9KRZIkqXXDngr4lVW1R5JrAarqgeb8/pIkaQ4adgTgF0nmAQWQZAxY2VpVkiSpVcMGgM8C\nXwOen+SjwOXAx1qrSpIktWqoXQBVdUaSq+ldxS/AIVV1Q6uVSZKk1qw1ADRD/0uralfgxvZLkiRJ\nbVvrLoCqegK4Kcn2s1CPJEmaBcN+C2BLYGmSK4FHV82sqte3UpUkSWrVsAHgT1qtQpIkzaphDwL8\n9lQ30BxDsBi4q6pel2RH4ExgK+Bq4M1V9fhU1y9JkiZv2FMB753kqiSPJHk8yRNJHhpyG+8B+r8x\n8EnghKraCXgAOHpyJUuSpOka9jwAn6N3GuCbgY2BtzLEqYCTbAf8BnBKMx1gf+CcZpHTgUMmV7Ik\nSZquYY8BoKqWJZnXfCvgS81pgT+wlmafAf4IeHYzvRXwYFWtaKaXA9tOsmZJkjTB1Vdf/fz58+ef\nAryUp3/AXwn8aMWKFW/dc88971s1c9gA8NPm3P9LknwKuJu1jB4keR1wX1VdnWS/yTyIpv0iYBHA\n9tv7DURpXWeflUZr/vz5p7zgBS/4T2NjYw9ssMEGtWr+ypUrMz4+vts999xzCvDkt/eG3QXw5mbZ\nd9L7GuCLgN9ZS5t9gdcnuY3eQX/7AycCWyRZFTy2A+4a1LiqTq6qBVW1YGxsbMgyJY2KfVYauZeO\njY091P/PH2CDDTaosbGxn9AbGXjSGkcAkmxfVXdU1e3NrJ8Dxw9TRVV9gGYXQTMC8L6qemOSrwCH\n0gsFRwLnDbO+YZ100ZKZXJ36vO2g3UddgiRp9TaY+M+/745iwof+tY0AfH3VjSTnTr82AN4PHJNk\nGb1jAk6dofVKkqQhre0YgPTdfvFUN1JVlwGXNbdvAfaa6rokSdL0rW0EoFZzW5IkrVtWrly5Mqu5\nI/S+DfCktQWAlyd5KMnDwMua2w8leXgSJwKSJEnt+9H4+PhzJoaA5lsAzwF+1D9/jbsAqmpeCwVK\nkqQZtmLFirfec889p9xzzz2rPQ9A//JDnwhIkiStu5qT/Ax9ld5hzwMgSZLWIwYASZI6yAAgSVIH\nGQAkSeogA4AkSR1kAJAkqYMMAJIkdZABQJKkDjIASJLUQQYASZI6yAAgSVIHGQAkSeogA4AkSR3U\nWgBI8qIklya5PsnSJO9p5j83ycVJbm5+b9lWDZIkabA2RwBWAH9YVbsBewPvSLIbcCxwSVXtDFzS\nTEuSpFnUWgCoqrur6prm9sPADcC2wMHA6c1ipwOHtFWDJEkabFaOAUiyA/AK4Apg66q6u7nrHmDr\n2ahBkiQ9pfUAkGQz4FzgD6rqof77qqqAWk27RUkWJ1k8Pj7edpmSpsk+K80trQaAJM+i98//jKr6\najP73iTbNPdvA9w3qG1VnVxVC6pqwdjYWJtlSpoB9llpbmnzWwABTgVuqKpP9911PnBkc/tI4Ly2\napAkSYPNb3Hd+wJvBq5LsqSZ98fAJ4CzkxwN3A4c1mINkiRpgNYCQFVdDmQ1dx/Q1nYlSdLatTkC\nIA3lpIuWrH0hTdnbDtp91CVIWgd5KmBJkjrIACBJUgcZACRJ6iADgCRJHWQAkCSpgwwAkiR1kAFA\nkqQOMgBIktRBBgBJkjrIACBJUgcZACRJ6iADgCRJHWQAkCSpgwwAkiR1kAFAkqQOMgBIktRBIwkA\nSRYmuSnJsiTHjqIGSZK6bNYDQJJ5wOeB1wK7AUck2W2265AkqctGMQKwF7Csqm6pqseBM4GDR1CH\nJEmdNYoAsC1wZ9/08maeJEmaJfNHXcDqJFkELGomH0ly0yjradHzgPtHXcQw3j7qAtYdc+Y1g0m/\nbhdW1cKpbMc+u26y3z5pzrxus9Znq2oq7aYsyT7AcVX16830BwCq6uOzWsg6Isniqlow6jo0PF+z\nbvP1n5t83Z5pFLsArgJ2TrJjkg2BNwDnj6AOSZI6a9Z3AVTViiTvBL4FzAO+WFVLZ7sOSZK6bCTH\nAFTVN4FvjmLb66CTR12AJs3XrNt8/ecmX7cJZv0YAEmSNHqeCliSpA4yAEiS1EEGAEmSOsgAIElS\nBxkAJEnqIAOAJEkdZACQJKmDDACSJHWQAUCSpA4yAEiS1EEGAEmSOsgAIElSBxkApijJE0mW9P0c\nO4m2+yW5YJrbvyzJgim2nfb2m/VsmOQzSZY1Pxck2X41y56WZL8B87du2v0gyfVJZuwqkUlOSbLb\nDKznqCSfm4H17Jnkuua5+mySTHedGp591j47hfV8NMmdSR6Z7rrWRSO5HPB64mdVtfsoNpxk3ii2\nO8DHgGcDu1TVE0neApyXZM+qWjnkOv4MuLiqTgRI8rLJFJBkXlU9Mei+qnrrZNY1C74A/D5wBb3L\nYS8E/mmkFXWLfdY+O1n/G/gccPOoC2mDIwAzLMltST7efMJYnGSPJN9K8u9J3t636OZJvpHkpiR/\nk2SDpv0XmnZLkxw/Yb2fTHIN8Lt98zdokvpHmumDknwvyTVJvpJks2b+wiQ3Nu1/ewYe5ybAW4D3\nrurMVfUl4BHgwEmsahtg+aqJqvphs/6nfeJJ8rkkRzW3+5+L/y/JlX3L7ZDkuub2ZUkWJHl7kr/o\nW+bJTwdJ3pTkyub1OmnVG3WStyT5t2bd+07qyRkgyTbA5lX1/epdg/vvgEOmu15Nn33WPrs6TX+9\neybWtS4yAEzdxnn6cOLhfffd0XzS+FfgNOBQYG/g+L5l9gLeBewGvISnOvgHq2oB8DLg1RPS9Y+r\nao+qOrOZng+cAdxcVR9K8jzgQ8CBVbUHsBg4JslGwN8CvwnsCbxg0ANKssuEx9T/s8WExXdqHudD\nE+Yvbh7TsD4PnJrk0iQfTPLCIdutei4+AWyYZMdm/uHAWROWPRf4rb7pw4Ezk/yn5va+zev1BPDG\n5p/18fRtkG59AAAQBklEQVTeRH5tdY8nyWtW81x9d8Di29L3ptnc3nbIx6qZYZ+1z06mz6733AUw\ndWsaTjy/+X0dsFlVPQw8nOSxvk55ZVXdApDky/T+aM8BDkuyiN5rsw29P+QfNm0mdpKTgLOr6qPN\n9N7N8t9Jb/fyhsD3gF2BW6vq5mZ7/wAsmlh0Vd0EzOoQaVV9K8mL6Q2Hvxa4NslLh2ja/1ycTe9N\n4RPN7/43dqpqPMktSfamN5S3K/Ad4B303lyvap6vjYH7gFcCl1XVOECSs4BfHlD7pczy86Vpsc/O\nAPvs+sMA0I7Hmt8r+26vml71nNeENtUk4vcBv1pVDyQ5Ddiob5lHJ7T5LvCaJH9VVT8HQm/f3BH9\nCyUZ6g8+yS488w1rlf2q6sG+6X8Htk/y7ObNcpU96aX3oVXVfwD/CPxjM4T4KuBenj5CtdGEZv3P\nxVnAV5J8tbe6GrS/7kzgMOBG4GtVVem9g5xeVR/oXzDJUEPzSV4DnDDgrp9W1X+ZMO8uYLu+6e2a\neVo32GcnoSN9dr3nLoDR2SvJjuntRzwcuBzYnF4n+UmSreml6zU5ld7BZGcnmQ98H9g3yU4ASTZN\n8sv0OtAOSV7StDti0Mqq6qaq2n01Pw9OWPZR4HTg03374H4P+Dm9pD6UJPunt2+SJM+mN7R6B3A7\nsFuSX2o+gR2wunVU1b/TGwr8E1b/Zvg14ODmsa8ajr0EODTJ85vtPzfJ/0PvIL1XJ9kqybPo2387\nYbuXrua5esYbSbMf8aEkezdvYr8HnLem50brHPss3emzXeAIwNRtnGRJ3/SFVTX014qAq+gdXboT\ncCm9hLsyybX0Ov+dDNEpq+rTSZ4D/D3wRuAo4MtJfqlZ5ENV9W/NEOU3kvyU3n7OZ0+i1tX5APAX\nwE1JNgbGgX2ag9yGtSfwuSQr6AXSU6rqKoAkZwM/Am4Frl3Les5qatlx0J3Np7MbgN2q6spm3vVJ\nPgRc1Lyp/wJ4R1V9P8lx9IZiHwSWDFrnFPxPevuXN6Z39L/fAJhd9ln77KQk+RTw34FNkiyn91iP\nm4l1rwsyudddGizJC+j9Q/tCVZ084P7TgNOq6rJZLk3SAPZZOQKgGVFV9wCvGHUdkoZjn5XHAGi2\nfB24bdRFSBqafXY95y4ASZI6yBEASZI6aE4cA7Bw4cK68MILR12G1AUzcoEi+6w0a6bcZ+fECMD9\n998/6hIkTYJ9Vlr3zYkAIEmSZpYBQJKkDjIASJLUQQYASZI6qNUAkOS9SZYm+VGSLyfZqLmYxhVJ\nliU5K8mGbdYgSZKeqbUAkGRb4N3Agqp6KTAPeAPwSeCEqtoJeAA4uq0aJEnSYG3vAphP7wpc84FN\ngLuB/YFzmvtPB4a6jrMkSZo5rQWAqroL+Et614m+G/gJcDXwYFWtaBZbDmzbVg2SJGmwNncBbAkc\nTO9azy8ENgUWTqL9oiSLkyweHx9vqUpJM8U+K80tbe4COBC4tarGq+oXwFeBfYEtml0CANsBdw1q\nXFUnV9WCqlowNjbWYpmSZoJ9Vppb2gwAdwB7J9kkSYADgOuBS4FDm2WOBM5rsQZJkjRAm8cAXEHv\nYL9rgOuabZ0MvB84JskyYCvg1LZqkCRJg7V6NcCq+jDw4QmzbwH2anO7kiRpzTwToCRJHWQAkCSp\ngwwAkiR1kAFAkqQOMgBIktRBBgBJkjrIACBJUgcZACRJ6iADgCRJHWQAkCSpgwwAkiR1kAFAkqQO\nMgBIktRBBgBJkjrIACBJUgcZACRJ6iADgCRJHWQAkCSpg1oNAEm2SHJOkhuT3JBknyTPTXJxkpub\n31u2WYMkSXqmtkcATgQurKpdgZcDNwDHApdU1c7AJc20JEmaRa0FgCTPAV4FnApQVY9X1YPAwcDp\nzWKnA4e0VYMkSRqszRGAHYFx4EtJrk1ySpJNga2r6u5mmXuArVusQZIkDdBmAJgP7AF8oapeATzK\nhOH+qiqgBjVOsijJ4iSLx8fHWyxT0kywz0pzS5sBYDmwvKquaKbPoRcI7k2yDUDz+75Bjavq5Kpa\nUFULxsbGWixT0kywz0pzS2sBoKruAe5Msksz6wDgeuB84Mhm3pHAeW3VIEmSBpvf8vrfBZyRZEPg\nFuAt9ELH2UmOBm4HDmu5BkmSNEGrAaCqlgALBtx1QJvblSRJa+aZACVJ6iADgCRJHWQAkCSpgwwA\nkiR1kAFAkqQOMgBIktRBBgBJkjpoqACQnjcl+dNmevske7VbmiRJasuwIwB/DewDHNFMPwx8vpWK\nJElS64Y9E+Arq2qPJNcCVNUDzel9JUnSHDTsCMAvksyjuXRvkjFgZWtVSZKkVg0bAD4LfA14fpKP\nApcDH2utKkmS1KqhdgFU1RlJrqZ3EZ8Ah1TVDa1WJkmSWrPWANAM/S+tql2BG9svSZIktW2tuwCq\n6gngpiTbz0I9kiRpFgz7LYAtgaVJrgQeXTWzql7fSlWSJKlVwwaAP2m1CkmSNKuGPQjw21PdQHMM\nwWLgrqp6XZIdgTOBrYCrgTdX1eNTXb8kSZq8YU8FvHeSq5I8kuTxJE8keWjIbbwH6P/GwCeBE6pq\nJ+AB4OjJlSxJkqZr2PMAfI7eaYBvBjYG3soQpwJOsh3wG8ApzXSA/YFzmkVOBw6ZXMmSJGm6hr4a\nYFUtA+ZV1RNV9SVg4RDNPgP8EU+dNXAr4MGqWtFMLwe2nUS9kiRpBgwbAH7anPt/SZJPJXnv2tom\neR1wX1VdPZXCkixKsjjJ4vHx8amsQtIsss9Kc8uwAeDNzbLvpPc1wBcBv7OWNvsCr09yG72D/vYH\nTgS2SLLq4MPtgLsGNa6qk6tqQVUtGBsbG7JMSaNin5XmlrV9it8eoKpur6qfV9VDVXV8VR3T7BJY\nrar6QFVtV1U7AG8A/rmq3ghcChzaLHYkcN60H4UkSZqUtY0AfH3VjSTnztA23w8ck2QZvWMCTp2h\n9UqSpCGt7TwA6bv94qlupKouAy5rbt8C7DXVdUmSpOlb2whArea2JEmaw9Y2AvDy5oQ/ATbuO/lP\ngKqqzVutTpIktWKNAaCq5s1WIZIkafYMfSIgSZK0/jAASJLUQQYASZI6yAAgSVIHGQAkSeogA4Ak\nSR1kAJAkqYMMAJIkdZABQJKkDjIASJLUQQYASZI6yAAgSVIHGQAkSeogA4AkSR3UWgBI8qIklya5\nPsnSJO9p5j83ycVJbm5+b9lWDZIkabA2RwBWAH9YVbsBewPvSLIbcCxwSVXtDFzSTEuSpFnUWgCo\nqrur6prm9sPADcC2wMHA6c1ipwOHtFWDJEkabFaOAUiyA/AK4Apg66q6u7nrHmDr2ahBkiQ9pfUA\nkGQz4FzgD6rqof77qqqAWk27RUkWJ1k8Pj7edpmSpsk+K80trQaAJM+i98//jKr6ajP73iTbNPdv\nA9w3qG1VnVxVC6pqwdjYWJtlSpoB9llpbmnzWwABTgVuqKpP9911PnBkc/tI4Ly2apAkSYPNb3Hd\n+wJvBq5LsqSZ98fAJ4CzkxwN3A4c1mINkiRpgNYCQFVdDmQ1dx/Q1nYlSdLaeSZASZI6yAAgSVIH\nGQAkSeqgNg8CHImTLlqy9oU0JW87aPdRlyBJmiHrXQCQNLcY2ttlcNfqGAAkSZNmcGvPbIU2jwGQ\nJKmDHAHQyPlJol0OAUsaxBEASZI6yAAgSVIHGQAkSeogA4AkSR1kAJAkqYMMAJIkdZABQJKkDjIA\nSJLUQQYASZI6yAAgSVIHjSQAJFmY5KYky5IcO4oaJEnqslkPAEnmAZ8HXgvsBhyRZLfZrkOSpC4b\nxQjAXsCyqrqlqh4HzgQOHkEdkiR11igCwLbAnX3Ty5t5kiRplqyzlwNOsghY1Ew+kuSmUdbToucB\n94+6iGG8fdQFrDvmzGsGk37dLqyqhVPZjn123WS/fdKced1mrc9W1VTaTVmSfYDjqurXm+kPAFTV\nx2e1kHVEksVVtWDUdWh4vmbd5us/N/m6PdModgFcBeycZMckGwJvAM4fQR2SJHXWrO8CqKoVSd4J\nfAuYB3yxqpbOdh2SJHXZSI4BqKpvAt8cxbbXQSePugBNmq9Zt/n6z02+bhPM+jEAkiRp9DwVsCRJ\nHWQAmKYk705yQ5IzWlr/cUne18a6NTOS7JfkglHXoeHYZ2Wf7VlnzwMwh/xP4MCqWj7qQiQNxT4r\n4QjAtCT5G+DFwD8l+WCSLya5Msm1SQ5uljkqydeTXJzktiTvTHJMs8z3kzy3We73k1yV5AdJzk2y\nyYDtvSTJhUmuTvKvSXad3Ue8/kqyQ5Ibk5yW5N+SnJHkwCTfSXJzkr2an+81r913k+wyYD2bDvo7\n0LrBPrv+sM/OgKryZxo/wG30zjD1MeBNzbwtgH8DNgWOApYBzwbGgJ8Ab2+WOwH4g+b2Vn3r/Ajw\nrub2ccD7mtuXADs3t18J/POoH//68gPsAKwAfoVeML4a+CIQeteq+DqwOTC/Wf5A4Nzm9n7ABc3t\ngX8Ho358/jzttbbPrgc/9tnp/7gLYOYcBLy+b9/fRsD2ze1Lq+ph4OEkPwH+dzP/OuBlze2XJvkI\nvT/AzeidJ+FJSTYD/gvwlSSrZv9SGw+kw26tqusAkiwFLqmqSnIdvTeb5wCnJ9kZKOBZA9axur+D\nG9ouXpNmn5377LPTYACYOQF+p6qedv7zJK8EHuubtbJveiVPvQanAYdU1Q+SHEUvofbbAHiwqnaf\n2bLVZ22v05/T+8fwW0l2AC4bsI6BfwdaJ9ln5z777DR4DMDM+RbwrjRRP8krJtn+2cDdSZ4FvHHi\nnVX1EHBrkt9t1p8kL59mzZqc5wB3NbePWs0y0/070Oyxz67/7LNrYACYOX9Ob3jph81Q1J9Psv2f\nAFcA3wFuXM0ybwSOTvIDYCm9/VyaPZ8CPp7kWlY/ejbdvwPNHvvs+s8+uwaeCVCSpA5yBECSpA4y\nAEiS1EEGAEmSOsgAIElSBxkAJEnqIAOApqw5l/rSJD9MsqQ5gYqkdZR9Vv08E6CmJMk+wOuAParq\nsSTPAzYccVmSVsM+q4kcAdBUbQPcX1WPAVTV/VX1/yfZM8m3m6uffSvJNknmN1dN2w8gyceTfHSU\nxUsdZJ/V03giIE1Jc6GTy4FNgP8DnAV8F/g2cHBVjSc5HPj1qvofSf4zcA7wLuAvgFdW1eOjqV7q\nHvusJnIXgKakqh5JsifwX4HX0Hsz+QjwUuDi5rTa84C7m+WXJvl74AJgH99IpNlln9VEBgBNWVU9\nQe/qWpc1l998B7C0qvZZTZNfAR4Enj87FUrqZ59VP48B0JQk2aW5xvYqu9O7fvZYc7ARSZ7VDCOS\n5LeB5wKvAv5Xki1mu2apy+yzmshjADQlzVDi/wK2AFYAy4BFwHbAZ+ldhnM+8Bnga/T2NR5QVXcm\neTewZ1UdOYrapS6yz2oiA4AkSR3kLgBJkjrIACBJUgcZACRJ6iADgCRJHWQAkCSpgwwAkiR1kAFA\nkqQOMgBIktRB/xckihIJeg5uAwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7c46410>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# grid = sns.FacetGrid(train_df, col='Embarked', hue='Survived', palette={0: 'k', 1: 'w'})\n",
    "grid = sns.FacetGrid(train_df, row='Embarked', col='Survived', size=2.2, aspect=1.6)\n",
    "grid.map(sns.barplot, 'Sex', 'Fare', alpha=.5, ci=None)\n",
    "grid.add_legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 处理数据\n",
    "\n",
    "- 根据前面的假设用删除特征的方法对数据集进行纠错构造补全处理。\n",
    "- 删除特征可以减少需要处理的数据量，加快分析。\n",
    "- 根据假设和决策，删除Cabin和Ticket\n",
    "- 注意保持训练集和测试集的一致性"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('Before', (891, 12), (418, 11), (891, 12), (418, 11))\n",
      "('After', (891, 10), (418, 9), (891, 10), (418, 9))\n"
     ]
    }
   ],
   "source": [
    "print(\"Before\", train_df.shape, test_df.shape, combine[0].shape, combine[1].shape)\n",
    "\n",
    "train_df = train_df.drop(['Ticket', 'Cabin'], axis=1)\n",
    "test_df = test_df.drop(['Ticket', 'Cabin'], axis=1)\n",
    "combine = [train_df, test_df]\n",
    "\n",
    "print(\"After\", train_df.shape, test_df.shape, combine[0].shape, combine[1].shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 从现有的特征中构建新特征\n",
    "在删除Name和PassengerId特征之前，抽取头衔titles特征，并测试其和存活特征之间的相关性\n",
    "\n",
    "使用正则表达式抽取titles。正则模式匹配**(\\w+\\.)**，以一个大写字母开头，点结尾的子字符串。expand=False表示返回一个DataFrame\n",
    "\n",
    "### Observations\n",
    "1. 大多数头衔准确对应年龄区间，比如说Master对应平均5岁\n",
    "2. 不同头衔的Age存活率相差不大\n",
    "3. 有些头衔大多数都存活(Mme, Lady, Sir)，有些大多数都死亡(Don, Rev, Jonkheer)\n",
    "\n",
    "### Decision\n",
    "1. 把Title加入到模型中"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>S</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch     Fare Embarked  Title  \n",
       "0      0   7.2500        S      1  \n",
       "1      0  71.2833        C      3  \n",
       "2      0   7.9250        S      2  \n",
       "3      0  53.1000        S      3  \n",
       "4      0   8.0500        S      1  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for dataset in combine:\n",
    "    dataset['Title'] = dataset.Name.str.extract(' ([A-Za-z]+)\\.', expand=False)\n",
    "\n",
    "pd.crosstab(train_df['Title'], train_df['Sex'])\n",
    "\n",
    "# 把一些出现次数少的title集合到一起，命名为rare\n",
    "for dataset in combine:\n",
    "    dataset['Title'] = dataset['Title'].replace(['Lady', 'Countess','Capt', 'Col',\\\n",
    " \t'Don', 'Dr', 'Major', 'Rev', 'Sir', 'Jonkheer', 'Dona'], 'Rare')\n",
    "\n",
    "    dataset['Title'] = dataset['Title'].replace('Mlle', 'Miss')\n",
    "    dataset['Title'] = dataset['Title'].replace('Mme', 'Mrs')\n",
    "    \n",
    "train_df[['Title', 'Survived']].groupby(['Title'], as_index=False).mean()\n",
    "\n",
    "# 把类别特征转化为序数\n",
    "title_mapping = {\"Mr\": 1, \"Miss\": 2, \"Mrs\": 3, \"Master\": 4, \"Rare\": 5}\n",
    "for dataset in combine:\n",
    "    dataset['Title'] = dataset['Title'].map(title_mapping)\n",
    "    dataset['Title'] = dataset['Title'].fillna(0)\n",
    "\n",
    "train_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 删除Name和PassagerId特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((891, 9), (418, 9))"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df = train_df.drop(['Name', 'PassengerId'], axis=1)\n",
    "test_df = test_df.drop(['Name'], axis=1)\n",
    "combine = [train_df, test_df]\n",
    "train_df.shape, test_df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 转化类别特征\n",
    "- 因为很多算法要求，把一些字符型的特征转化为数值型的值。\n",
    "- 把Sex进行转化，female=1，male=0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>S</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Survived  Pclass  Sex   Age  SibSp  Parch     Fare Embarked  Title\n",
       "0         0       3    0  22.0      1      0   7.2500        S      1\n",
       "1         1       1    1  38.0      1      0  71.2833        C      3\n",
       "2         1       3    1  26.0      0      0   7.9250        S      2\n",
       "3         1       1    1  35.0      1      0  53.1000        S      3\n",
       "4         0       3    0  35.0      0      0   8.0500        S      1"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for dataset in combine:\n",
    "    dataset['Sex'] = dataset['Sex'].map( {'female': 1, 'male': 0} ).astype(int)\n",
    "\n",
    "train_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 补全连续数值型特征\n",
    "\n",
    "首先补全Age特征\n",
    "\n",
    "可选方法：\n",
    "1. 用同均值方差的随机数补全\n",
    "2. 用其他相关的特征（Age, Gender, and Pclass）进行分类，然后取中位数进行补全\n",
    "3. 综合1和2，先分类，然后取均值方差随机数进行补全\n",
    "\n",
    "- 1和3会带来新的噪声，所以选择2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>38</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>35</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>S</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Survived  Pclass  Sex  Age  SibSp  Parch     Fare Embarked  Title\n",
       "0         0       3    0   22      1      0   7.2500        S      1\n",
       "1         1       1    1   38      1      0  71.2833        C      3\n",
       "2         1       3    1   26      0      0   7.9250        S      2\n",
       "3         1       1    1   35      1      0  53.1000        S      3\n",
       "4         0       3    0   35      0      0   8.0500        S      1"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgAAAAHUCAYAAABMP5BeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm0ZWV95vHvQxU4QYNgBWmKNChIBOIQShxj2xiliEZI\nQhSbIKzGIL0kcYxCjGmNGnGZBaIxvSSgoIGIKEYWoSHIIDiVFDLIEIUgKoSSKsUpMZGyfv3H2QU3\n11vc6Uz3vN/PWnfds/fZw++tU++5z373PvukqpAkSW3ZatQFSJKk4TMASJLUIAOAJEkNMgBIktQg\nA4AkSQ0yAEiS1CADgCRJDTIADFGSnye5PslNSc5L8siHWPZtSd44zPq2UMevJPlSkv94qHqSnJnk\neTPM3znJhUluSHJLkosGWvDMtT0syblJbk+yJsnuw65Bk8f+PLL+/NwkX02yMclhw97/JDEADNdP\nq+opVbUf8DPguFEXNAffB/4I+MsFrv/nwKVV9eSq2gc4oW+Vzd0xwH1VtSdwCvCeEdSgyWN/Hk1/\n/jZwNHDOCPY9UQwAo3M1sCdAklckubFL1R+bvmCSP0hyTff8pzYfaST5ve7o44YkV3Xz9k3yle7I\n5MYkey2myKq6t6quAe5f4CZ2Ae6asr0bNz9O8sddu25M8vZu3tO66YcneVSSm5Pst5g2AIcAZ3WP\nPwk8P0kWuU1pKvvzkPpzVd3Z7XfTYrYjWD7qAlqUZDlwMHBxkn2BPwWeVVUbkuw4wyrnV9XfdOu+\nk94R7QeAPwMOqqq7k+zQLXsccGpVnZ1kG2DZDPs/F9h7hv2cXFUfXWz7pvkgcG6S44HPAh+pqn9J\n8kJgL+AAIMAFSZ5bVVcluQB4J/AI4G+r6qYZ2nA1sN0M+3tjVX122rxdge8AVNXGJD8EdgI29KeJ\napn9eej9WX1iABiuRyS5vnt8NXAG8CrgvKraAFBV359hvf26N4odgG2BS7r5XwDOTPIJ4Pxu3peA\ntyRZSe+N5rbpG6uql/WrQbOpqkuSPA5YTe9N8rruCOCF3c913aLb0nsDuYreMOM1wL/TG66cabu/\nPuDSpdnYn+3PS5oBYLh+WlVPmTpjjiPRZwKHVtUNSY4GngdQVccleTrwIuDaJPtX1TlJ1nTzLkry\nqqq6fNo+h3nEsPlN8BzgnCQXAs+ld5Tw7qr60Ayr7ETvDWRr4OHAv05fYJ5HDHcDuwF3dUdr2wPf\nW2BzpM3sz6Ppz+oTA8DoXQ58OsnJVfW9JDvOcNSwHXBPkq2BI+j9QSPJ46tqDbAmycHAbkm2B+6o\nqvcn+WXgSd0+HjDMI4YkBwJfrqp/S7Id8Hh6F/H8GHhHkrOr6idJdgXur6p7gQ8BbwX2oHfB3vHT\ntzvPI4YLgKPoHU0dBlxefg2mBsP+PPj+rD4xAIxYVd2c5F3A55L8nN4Q2tHTFnsrsAZY3/3enJTf\n210UFOAy4AbgzcCRSe4H1gF/sZj6kjwWWAv8F2BTktcC+1TVj+a4if2Bv0qykd5Fp6d3FyGR5InA\nl7qjpp8Av59kNb03jnOSLAO+mOTA6Uc983QG8LEkt9O7CvrwRWxL2iL78+D7c5KnAZ8GHg38VpK3\nV9W+C91ey+KBkPohyZnAmVV15YhLkbRI9uc2+DFASZIaZABQv/w9cOeoi5DUF/bnBngKQJKkBjkC\nIElSg4b6KYDVq1fXxRdfPMxdSvpFfbkNsv1ZGhsL6tNDHQHYsME7r0qTwv4sLW2eApAkqUEGAEmS\nGmQAkCSpQQYASZIaZACQJKlBBgBJkhpkAJAkqUEGAEmSGmQAkCSpQQYASZIaZACQJKlBBgBJkhpk\nAJAkqUEGAEmSGmQAkCSpQQYASZIaZACQJKlBBgBJkhpkAJAkqUEGAEmSGmQAkCSpQQYASZIaZACQ\nJKlBBgBJkhpkAJAkqUEGAEmSGmQAkCSpQXMOAEmWJbkuyYXd9B5J1iS5Pcm5SbYZXJmSJKmf5jMC\n8Brg1inT7wFOqao9gfuAY/pZmCRJGpw5BYAkK4EXAad30wEOBD7ZLXIWcOggCpQkSf031xGA9wFv\nAjZ10zsBP6iqjd30XcCufa5NkiQNyKwBIMmLgXur6tqF7CDJsUnWJlm7fv36hWxC0piwP0uTYy4j\nAM8GXpLkTuDj9Ib+TwV2SLK8W2YlcPdMK1fVaVW1qqpWrVixog8lSxoV+7M0OWYNAFV1YlWtrKrd\ngcOBy6vqCOAK4LBusaOAzwysSkmS1FeLuQ/Am4HXJ7md3jUBZ/SnJEmSNGjLZ1/kQVV1JXBl9/gO\n4ID+lyRJkgbNOwFKktQgA4AkSQ0yAEiS1CADgCRJDTIASJLUIAOAJEkNMgBIktQgA4AkSQ0yAEiS\n1CADgCRJDTIASJLUIAOAJEkNMgBIktQgA4AkSQ0yAEiS1CADgCRJDTIASJLUIAOAJEkNMgBIktQg\nA4AkSQ0yAEiS1CADgCRJDTIASJLUIAOAJEkNMgBIktQgA4AkSQ0yAEiS1CADgCRJDTIASJLUIAOA\nJEkNMgBIktQgA4AkSQ2aNQAk2S3JFUluSXJzktd083dMcmmS27rfjx58uZIkqR/mMgKwEXhDVe0D\nPAN4dZJ9gBOAy6pqL+CyblqSJC0BswaAqrqnqr7aPf4xcCuwK3AIcFa32FnAoYMqUpIk9de8rgFI\nsjvwVGANsHNV3dM9tQ7Yua+VSZKkgZlzAEiyLfAp4LVV9aOpz1VVAbWF9Y5NsjbJ2vXr1y+qWEmj\nZX+WJsecAkCSren98T+7qs7vZn83yS7d87sA9860blWdVlWrqmrVihUr+lGzpBGxP0uTYy6fAghw\nBnBrVZ085akLgKO6x0cBn+l/eZIkaRCWz2GZZwNHAl9Lcn0370+Ak4BPJDkG+Bbw0sGUKEmS+m3W\nAFBVnweyhaef399yJEnSMHgnQEmSGmQAkCSpQQYASZIaZACQJKlBBgBJkhpkAJAkqUEGAEmSGmQA\nkCSpQXO5E6C0IKdc+o05Lfe6FzxhwJVIkqZzBECSpAYZACRJapABQJKkBhkAJElqkAFAkqQG+SmA\nMdPvK+e9El+SNBNHACRJapAjAALmPlIA/R8tcJRCkobPEQBJkhrkCIAkLSEPNWLmKJnmwxEASZIa\n5AiAJM3DbNesLPYofD7X40iL4QiAJEkNcgRgifIoQRpPgx4hWIxxrk3D5wiAJEkNcgRAS8Yg7lXg\nPQjaM8mjZ6NumyMMS4sjAJIkNcgRgCEZdTKXJGkqRwAkSWqQAUCSpAZ5CmARWh3WXwrtXgo1SuNm\n1Dc58iLB4XIEQJKkBjkCIPWRHyuUFm6QI3f2uV/kCIAkSQ1a1AhAktXAqcAy4PSqOqkvVTGYm75I\nC+U1BZL9YNIseAQgyTLgg8DBwD7Ay5Ps06/CJEnS4CxmBOAA4PaqugMgyceBQ4Bb+lHYIHh+VkvR\nJP6/HeTV4ON+lDru9akdi7kGYFfgO1Om7+rmSZKkMZeqWtiKyWHA6qp6ZTd9JPD0qjp+2nLHAsd2\nk3sDX59l048BNiyoqPE0Se2ZpLZAu+3ZUFWrF7KDBfTn+dS1FExSW8D2jLP5tGVBfXoxAeCZwNuq\n6qBu+kSAqnr3gjb44HbXVtWqxWxjnExSeyapLWB7hmVc61qISWoL2J5xNoy2LOYUwDXAXkn2SLIN\ncDhwQX/KkiRJg7TgiwCramOS44FL6H0M8MNVdXPfKpMkSQOzqPsAVNVFwEV9qmWz0/q8vVGbpPZM\nUlvA9gzLuNa1EJPUFrA942zgbVnwNQCSJGnp8lbAkiQ1yAAgSVKDDACSJDXIACBJUoMMAJIkNcgA\nIElSgwwAkiQ1yAAgSVKDDACSJDXIACBJUoMMAJIkNcgAMERJfp7k+iQ3JTkvySMfYtm3JXnjMOvb\nQh1HJLkxydeSfDHJk7ew3JlJnjfD/J2TXJjkhiS3JOn3l0fNKsnDkpyb5PYka5LsPuwaNHnszyPr\nz89N8tUkG5McNuz9TxIDwHD9tKqeUlX7AT8Djht1QXPwTeC/V9WvAu9g/t9Q9efApVX15KraBzih\n3wXOwTHAfVW1J3AK8J4R1KDJY38eTX/+NnA0cM4I9j1RDACjczWwJ0CSV3Sp/IYkH5u+YJI/SHJN\n9/ynNh9pJPm97ujjhiRXdfP2TfKV7sjkxiR7LabIqvpiVd3XTX4ZWDnPTewC3DVlezdOadcfd+26\nMcnbu3lP66YfnuRRSW5Ost9i2gAcApzVPf4k8PwkWeQ2pansz0Pqz1V1Z7ffTYvZjmD5qAtoUZLl\nwMHAxUn2Bf4UeFZVbUiy4wyrnF9Vf9Ot+056R7QfAP4MOKiq7k6yQ7fsccCpVXV2km2AZTPs/1xg\n7xn2c3JVffQhSj8G+H9za+UDPgicm+R44LPAR6rqX5K8ENgLOAAIcEGS51bVVUkuAN4JPAL426q6\naYY2XA1sN8P+3lhVn502b1fgOwBVtTHJD4GdgA3zbIv0C+zPQ+/P6hMDwHA9Isn13eOrgTOAVwHn\nVdUGgKr6/gzr7de9UewAbAtc0s3/AnBmkk8A53fzvgS8JclKem80t03fWFW9bL6FJ/kf9N4wnjOf\n9arqkiSPA1bTe5O8rjsCeGH3c1236Lb03kCuojfMeA3w78AfbWG7vz7fNkh9Zn+2Py9pBoDh+mlV\nPWXqjDmORJ8JHFpVNyQ5GngeQFUdl+TpwIuAa5PsX1XnJFnTzbsoyauq6vJp+5zXEUOSJwGnAwdX\n1ffmUvBU3ZvgOcA5SS4EnkvvKOHdVfWhGVbZid4byNbAw4F/naGm+Rwx3A3sBtzVHa1tD8y7HdI0\n9ufR9Gf1iQFg9C4HPp3k5Kr6XpIdZzhq2A64J8nWwBH0/qCR5PFVtQZYk+RgYLck2wN3VNX7k/wy\n8KRuHw+YzxFDt43zgSOr6hvzbVySA4EvV9W/JdkOeDy9i3h+DLwjydlV9ZMkuwL3V9W9wIeAtwJ7\n0Ltg7/jp253nEcMFwFH0jqYOAy6vqppvW6Q5sD8Pvj+rTwwAI1ZVNyd5F/C5JD+nN4R29LTF3gqs\nAdZ3vzcn5fd2FwUFuAy4AXgzcGSS+4F1wF8sssQ/o5fg/7o7utlYVavmsf7+wF8l2UjvotPTq+oa\ngCRPBL7UbfcnwO8nWU3vjeOcJMuALyY5cPpRzzydAXwsye3A94HDF7EtaYvsz4Pvz0meBnwaeDTw\nW0neXlX7LnR7LYsHQuqHJGcCZ1bVlSMuRdIi2Z/b4McAJUlqkAFA/fL3wJ2jLkJSX9ifG+ApAEmS\nGuQIgCRJDTIASJLUoKF+DHD16tV18cUXD3OXkn5RX74Hwf4sjY0F9emhjgBs2OCt16VJYX+WljZP\nAUiS1CADgCRJDTIASJLUIAOAJEkNMgBIktQgA4AkSQ0yAEiS1CADgCRJDTIASJLUIAOAJEkNMgBI\nktQgA4AkSQ0yAEiS1CADgCRJDTIASJLUIAOAJEkNMgBIktQgA4AkSQ0yAEiS1CADgCRJDTIASJLU\nIAOAJEkNMgBIktQgA4AkSQ0yAEiS1CADgCRJDZpzAEiyLMl1SS7spvdIsibJ7UnOTbLN4MqUJEn9\nNJ8RgNcAt06Zfg9wSlXtCdwHHNPPwiRJ0uDMKQAkWQm8CDi9mw5wIPDJbpGzgEMHUaAkSeq/uY4A\nvA94E7Cpm94J+EFVbeym7wJ27XNtkiRpQGYNAEleDNxbVdcuZAdJjk2yNsna9evXL2QTksaE/Vma\nHHMZAXg28JIkdwIfpzf0fyqwQ5Ll3TIrgbtnWrmqTquqVVW1asWKFX0oWdKo2J+lyTFrAKiqE6tq\nZVXtDhwOXF5VRwBXAId1ix0FfGZgVUqSpL5azH0A3gy8Psnt9K4JOKM/JUmSpEFbPvsiD6qqK4Er\nu8d3AAf0vyRJkjRo3glQkqQGGQAkSWqQAUCSpAYZACRJapABQJKkBhkAJElqkAFAkqQGGQAkSWqQ\nAUCSpAYZACRJapABQJKkBhkAJElqkAFAkqQGGQAkSWqQAUCSpAYZACRJapABQJKkBhkAJElqkAFA\nkqQGGQAkSWqQAUCSpAYZACRJapABQJKkBhkAJElqkAFAkqQGLR91AZIkafGuvfbaX1q+fPnpwH78\n5wP8TcBNGzdufOX+++9/7+aZBgBJkibA8uXLT3/sYx/7xBUrVty31VZb1eb5mzZtyvr16/dZt27d\n6cBLNs/3FIAkSZNhvxUrVvxo6h9/gK222qpWrFjxQ3ojAw/OH2ppkiRpULaa/sd/yhPFtL/5BgBJ\nkhpkAJAkqUEGAEmSJsOmTZs2ZQtPhN6nAR5gAJAkaTLctH79+u2nh4DuUwDbAzdNnT/rxwCT7AZ8\nFNgZKOC0qjo1yY7AucDuwJ3AS6vqvr40QZIkzcvGjRtfuW7dutPXrVu3xfsATF1+LvcB2Ai8oaq+\nmmQ74NoklwJHA5dV1UlJTgBOAN7cl1ZIkqR56W7y85JZF+zMegqgqu6pqq92j38M3ArsChwCnNUt\ndhZw6LyrlSRJIzGvawCS7A48FVgD7FxV93RPraN3ikCSJC0Bcw4ASbYFPgW8tqp+NPW5qip61wfM\ntN6xSdYmWbt+/fpFFStptOzP0uSYUwBIsjW9P/5nV9X53ezvJtmle34X4N6Z1q2q06pqVVWtWrFi\nRT9qljQi9mdpcswaAJIEOAO4tapOnvLUBcBR3eOjgM/0vzxJkjQIc/kUwLOBI4GvJbm+m/cnwEnA\nJ5IcA3wLeOlgSpQkSf02awCoqs8DM95ZCHh+f8uRJEnD4J0AJUlqkAFAkqQGGQAkSWqQAUCSpAYZ\nACRJapABQJKkBhkAJElqkAFAkqQGGQAkSWrQXG4FrCE75dJvPOTzr3vBExa9jbluR5I0mRwBkCSp\nQY4ALEFzObqXJOmhOAIgSVKDHAGQpCF7qFE8r83RsDgCIElSgwwAkiQ1yFMAkjQAXqyrcecIgCRJ\nDXIEYMg8KpAkjQNHACRJatDYjgB4K1tJkgbHEQBJkho0tiMAS5Hn9yUNkjcQUj85AiBJUoMcAZij\nSTy671ebPPKQpKXHEQBJkhrkCIAkjZFJHG3UeHIEQJKkBjkCoLHhvR/aMU5Xs49TLYux0JGDpdRG\n9ZcjAJIkNcgRAA1Fv85rOkow+cbpiLyF8/GDGjkYp9dRM3MEQJKkBi1qBCDJauBUYBlwelWd1Jeq\ntKSM21GS9zfQdOP2f1Rb5sjB8Cx4BCDJMuCDwMHAPsDLk+zTr8IkSdLgLGYE4ADg9qq6AyDJx4FD\ngFv6UVi/eM5YC+VIwvjx6HB8LKZ/OCIzHhZzDcCuwHemTN/VzZMkSWMuVbWwFZPDgNVV9cpu+kjg\n6VV1/LTljgWO7Sb3Br4+y6YfA2xYUFHjaZLaM0ltgXbbs6GqVi9kBwvoz/OpaymYpLaA7Rln82nL\ngvr0YgLAM4G3VdVB3fSJAFX17gVt8MHtrq2qVYvZxjiZpPZMUlvA9gzLuNa1EJPUFrA942wYbVnM\nKYBrgL2S7JFkG+Bw4IL+lCVJkgZpwRcBVtXGJMcDl9D7GOCHq+rmvlUmSZIGZlH3Aaiqi4CL+lTL\nZqf1eXujNkntmaS2gO0ZlnGtayEmqS1ge8bZwNuy4GsAJEnS0uWtgCVJapABQJKkBhkAJElqkAFA\nkqQGGQAkSWqQAUCSpAYZACRJapABQJKkBhkAJElqkAFAkqQGGQAkSWqQAUCSpAYZAIYoyc+TXJ/k\npiTnJXnkQyz7tiRvHGZ9W6jjkCQ3dnWvTfKcLSx3ZZLdZ5i/d/fc9UluTTL0b+tKsmOSS5Pc1v1+\n9LBr0OSxP4+sP/9ekpuTbEqyatj7nyQGgOH6aVU9par2A34GHDfqgubgMuDJVfUU4H8Bp89z/fcD\np3TtfiLwgX4XOAcnAJdV1V702nPCCGrQ5LE/j6Y/3wT8DnDVCPY9UQwAo3M1sCdAkld0qfyGJB+b\nvmCSP0hyTff8pzYfaXRJ+KZu/lXdvH2TfKVL6Dcm2WsxRVbVT+rB74x+FDDf74/eBbhryva+1tW5\nLMl7u3bdmORV3fzfTnJZenZJ8o0kj11MG4BDgLO6x2cBhy5ye9J09uch9eequrWqvr6Ybahn+agL\naFGS5cDBwMVJ9gX+FHhWVW1IsuMMq5xfVX/TrftO4Bh6yfvPgIOq6u4kO3TLHgecWlVnJ9kGWDbD\n/s8F9p5hPydX1UdnWP63gXcDvwS8aJ7NPQW4PMkXgX8EPlJVP+ja8MOqelqShwFfSPKPVfXpJL8L\nvBpYDfyfqlo3rZ7t6L3hzuR/VtUt0+btXFX3dI/XATvPsw3SFtmfh96f1ScGgOF6RJLru8dXA2cA\nrwLOq6oNAFX1/RnW2697o9gB2Ba4pJv/BeDMJJ8Azu/mfQl4S5KV9N5obpu+sap62XyKrqpPA59O\n8lzgHcBvzGPdjyS5hF7nPwR4VZInAy8EnpTksG7R7YG9gG8Cf0hvmO/LVfV3M2zzx8BT5tOGKetW\nkvke9UgzsT+PuD9rcQwAw/XT7tzbA5LMZb0zgUOr6oYkRwPPA6iq45I8nV6KvzbJ/lV1TpI13byL\nkryqqi6fts95HTFsVlVXJXlcksdsfoObi6r6F+DDwIeT3ATsBwT4w6q6ZIZVVgKbgJ2TbFVVm6bV\nP98jhu8m2aWq7kmyC3DvXGuXHoL9eTT9WX1iABi9y+ml8ZOr6ntJdpzhqGE74J4kWwNHAHcDJHl8\nVa0B1iQ5GNgtyfbAHVX1/iS/DDyp28cD5nPEkGRP4J+7I+dfAx4GfG8e66+mdwHe/d25v526+i8B\n/neSy7vnntDN/w96by4vB44CXg/85bT653vEcEG3rZO635+Zx7rSfNifB9+f1ScGgBGrqpuTvAv4\nXJKfA9cBR09b7K3AGmB993u7bv57u4uCQu/q3huANwNHJrmf3vnuv1hkib8LvKLb3k+Bl025iGgu\nXgicmuTfu+k/rqp1SU4Hdge+mt5h03p6F+e9Abi6qj6f5AbgmiT/UFW3LqINJwGfSHIM8C3gpYvY\nlrRF9ufB9+fuGoYPACuAf0hyfVUdtNDttSzze+2lmSW5Eji6qu4ccSmSFsn+3AY/BihJUoMMAOqX\nM4EfjLoISX1xJvbniecpAEmSGuQIgCRJDRrqpwBWr15dF1988TB3KekXzenD6rOxP0tjY0F9eqgj\nABs2zPleE5LGnP1ZWto8BSBJUoMMAJIkNcgAIElSgwwAkiQ1yO8CGIBTLv3GrMu87gVPGEIlkiTN\nzBEASZIaZACQJKlBBgBJkhpkAJAkqUEGAEmSGmQAkCSpQQYASZIaZACQJKlBBgBJkhpkAJAkqUEG\nAEmSGmQAkCSpQQYASZIaZACQJKlBBgBJkhpkAJAkqUEGAEmSGmQAkCSpQQYASZIaZACQJKlBBgBJ\nkhpkAJAkqUFzDgBJliW5LsmF3fQeSdYkuT3JuUm2GVyZkiSpn+YzAvAa4NYp0+8BTqmqPYH7gGP6\nWZgkSRqcOQWAJCuBFwGnd9MBDgQ+2S1yFnDoIAqUJEn9N9cRgPcBbwI2ddM7AT+oqo3d9F3Arn2u\nTZIkDcisASDJi4F7q+rahewgybFJ1iZZu379+oVsQtKYsD9Lk2MuIwDPBl6S5E7g4/SG/k8Fdkiy\nvFtmJXD3TCtX1WlVtaqqVq1YsaIPJUsaFfuzNDlmDQBVdWJVrayq3YHDgcur6gjgCuCwbrGjgM8M\nrEpJktRXi7kPwJuB1ye5nd41AWf0pyRJkjRoy2df5EFVdSVwZff4DuCA/pckSZIGzTsBSpLUIAOA\nJEkNMgBIktSgeV0DoP455dJvzGm5173gCQOuRJLUIkcAJElqkAFAkqQGGQAkSWqQAUCSpAYZACRJ\napABQJKkBhkAJElqkAFAkqQGGQAkSWqQAUCSpAYZACRJapDfBTAB/F4BSdJ8OQIgSVKDDACSJDXI\nACBJUoMMAJIkNcgAIElSgwwAkiQ1yAAgSVKDDACSJDXIACBJUoMMAJIkNchbAY+5ud7mV5Kk+XAE\nQJKkBhkAJElqkAFAkqQGeQ3APHg+XpI0KRwBkCSpQbMGgCS7JbkiyS1Jbk7ymm7+jkkuTXJb9/vR\ngy9XkiT1w1xGADYCb6iqfYBnAK9Osg9wAnBZVe0FXNZNS5KkJWDWAFBV91TVV7vHPwZuBXYFDgHO\n6hY7Czh0UEVKkqT+mtc1AEl2B54KrAF2rqp7uqfWATv3tTJJkjQwcw4ASbYFPgW8tqp+NPW5qiqg\ntrDesUnWJlm7fv36RRUrabTsz9LkmFMASLI1vT/+Z1fV+d3s7ybZpXt+F+DemdatqtOqalVVrVqx\nYkU/apY0IvZnaXLM5VMAAc4Abq2qk6c8dQFwVPf4KOAz/S9PkiQNwlxuBPRs4Ejga0mu7+b9CXAS\n8IkkxwDfAl46mBIlSVK/zRoAqurzQLbw9PP7W44kSRoG7wQoSVKDDACSJDXIACBJUoMMAJIkNciv\nA27IXL7O+HUveMIQKpEkjZojAJIkNcgAIElSgwwAkiQ1yAAgSVKDDACSJDVoLD8FMJer1cEr1gfB\nf3tJaoMjAJIkNWgsRwA0/ryngCbNXEe/tsT/71pqHAGQJKlBjgBI0hiYbQTCEQb1myMAkiQ1yAAg\nSVKDDACSJDXIawAkDdw4nN9e7FX+0qRxBECSpAZN/AiAd7aTNAyOMGipcQRAkqQGTfwIgKTx550l\npeFzBECSpAY5AqCB8foL9dM4fJJAmiSOAEiS1CBHADRynv+VBs8RFE3nCIAkSQ1yBKDjZ3glLWWD\nfg9zBGHyOAIgSVKDDACSJDVoSZ8CcNhekqSFcQRAkqQGLWoEIMlq4FRgGXB6VZ3Ul6qkCeNHHQdv\n0kcEJ719Gr4FjwAkWQZ8EDgY2Ad4eZJ9+lWYJEkanMWMABwA3F5VdwAk+ThwCHBLPwqTphrXox+P\n2jUpFtvHRv0xwVHvfylazDUAuwLfmTJ9VzdPkiSNuYF/CiDJscCx3eRPknx9llUeA2wYbFVDNUnt\nmaS2QB/a8/o+FdKnbc21PRdX1eqF7GAB/Xk+dS0Fk9QWGGJ7+tlXHsIW2zOk/ffTfF6bBfXpVNV8\n1+mtmDzvA18wAAAFZElEQVQTeFtVHdRNnwhQVe9e0AYf3O7aqlq1mG2Mk0lqzyS1BWzPsIxrXQsx\nSW0B2zPOhtGWxZwCuAbYK8keSbYBDgcu6E9ZkiRpkBZ8CqCqNiY5HriE3scAP1xVN/etMkmSNDCL\nugagqi4CLupTLZud1uftjdoktWeS2gK2Z1jGta6FmKS2gO0ZZwNvy4KvAZAkSUuXtwKWJKlBYxUA\nkqxO8vUktyc5YdT1zEeS3ZJckeSWJDcneU03f8cklya5rfv96FHXOh9JliW5LsmF3fQeSdZ0r9G5\n3QWgS0KSHZJ8Msk/Jbk1yTOX6uuT5HXd/7ObkvxdkoeP22uzlPszTGaftj+Pr1H06bEJABNwa+GN\nwBuqah/gGcCru/pPAC6rqr2Ay7rppeQ1wK1Tpt8DnFJVewL3AceMpKqFOZXe52V/BXgyvXYtudcn\nya7AHwGrqmo/ehfhHs4YvTYT0J9hMvu0/XkMjaxPV9VY/ADPBC6ZMn0icOKo61pEez4DvAD4OrBL\nN28X4Oujrm0ebVhJrxMdCFwIhN6NKZbP9JqN8w+wPfBNuutepsxfcq8PD96Fc0d6F/JeCBw0Tq/N\npPXnrg1Luk/bn8f3Z1R9emxGAJigWwsn2R14KrAG2Lmq7umeWgfsPKKyFuJ9wJuATd30TsAPqmpj\nN72UXqM9gPXAR7oh0NOTPIol+PpU1d3AXwLfBu4Bfghcy3i9NhPTn2Fi+rT9eUyNqk+PUwCYCEm2\nBT4FvLaqfjT1uerFuCXxsYskLwburaprR11LnywHfg34v1X1VOBfmTY8uFRen+685iH03gT/K/Ao\nYEG39tXsJqFP25/H26j69DgFgLuB3aZMr+zmLRlJtqb3RnF2VZ3fzf5ukl2653cB7h1VffP0bOAl\nSe4EPk5v2PBUYIckm+8fsZReo7uAu6pqTTf9SXpvIEvx9fkN4JtVtb6q7gfOp/d6jdNrs+T7M0xU\nn7Y/j7eR9OlxCgBL+tbCSQKcAdxaVSdPeeoC4Kju8VH0ziOOvao6sapWVtXu9F6Ly6vqCOAK4LBu\nsaXUnnXAd5Ls3c16Pr2vrl6Kr8+3gWckeWT3/25zW8bptVnS/Rkmq0/bn8feaPr0qC9+mHYhxG8C\n3wD+GXjLqOuZZ+3PoTfcdCNwfffzm/TOs10G3AZ8Fthx1LUuoG3PAy7sHj8O+ApwO3Ae8LBR1zeP\ndjwFWNu9Rn8PPHqpvj7A24F/Am4CPgY8bNxem6Xcn7v6J7JP25/H82cUfdo7AUqS1KBxOgUgSZKG\nxAAgSVKDDACSJDXIACBJUoMMAJIkNcgAoP8kyaFJKsmvjLoWSYtnn9aWGAA03cuBz3e/JS199mnN\nyACgB3T3PH8Ova+cPLybt1WSv+6+c/vSJBclOax7bv8kn0tybZJLNt+CU9J4sE/roRgANNUh9L5f\n+xvA95LsD/wOsDu973Q/kt5XUm6+R/oHgMOqan/gw8C7RlG0pC2yT2uLls++iBrycnpfEAK9Lwx5\nOb3/I+dV1SZgXZIruuf3BvYDLu3duppl9L7GUtL4sE9riwwAAiDJjvS+IexXkxS9zl/Ap7e0CnBz\nVT1zSCVKmgf7tGbjKQBtdhjwsar6b1W1e1XtBnwT+D7wu915w53pfZEIwNeBFUkeGD5Msu8oCpc0\nI/u0HpIBQJu9nF88MvgU8Fh63719C/C3wFeBH1bVz+i9wbwnyQ30vintWcMrV9Is7NN6SH4boGaV\nZNuq+kmSneh9NeWzq/d93JKWIPu0wGsANDcXJtkB2AZ4h28U0pJnn5YjAJIktchrACRJapABQJKk\nBhkAJElqkAFAkqQGGQAkSWqQAUCSpAb9f7Q7jl7BCpaPAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x80aac10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# grid = sns.FacetGrid(train_df, col='Pclass', hue='Gender')\n",
    "grid = sns.FacetGrid(train_df, row='Pclass', col='Sex', size=2.2, aspect=1.6)\n",
    "grid.map(plt.hist, 'Age', alpha=.5, bins=20)\n",
    "grid.add_legend()\n",
    "\n",
    "guess_ages = np.zeros((2,3))\n",
    "for dataset in combine:\n",
    "    # 计算中位数\n",
    "    for i in range(0, 2):\n",
    "        for j in range(0, 3):\n",
    "            guess_df = dataset[(dataset['Sex'] == i) & \\\n",
    "                                  (dataset['Pclass'] == j+1)]['Age'].dropna()\n",
    "\n",
    "            # age_mean = guess_df.mean()\n",
    "            # age_std = guess_df.std()\n",
    "            # age_guess = rnd.uniform(age_mean - age_std, age_mean + age_std)\n",
    "\n",
    "            age_guess = guess_df.median()\n",
    "\n",
    "            # Convert random age float to nearest .5 age\n",
    "            guess_ages[i,j] = int( age_guess/0.5 + 0.5 ) * 0.5\n",
    "    \n",
    "    # 补全\n",
    "    for i in range(0, 2):\n",
    "        for j in range(0, 3):\n",
    "            dataset.loc[ (dataset.Age.isnull()) & (dataset.Sex == i) & (dataset.Pclass == j+1),\\\n",
    "                    'Age'] = guess_ages[i,j]\n",
    "\n",
    "    dataset['Age'] = dataset['Age'].astype(int)\n",
    "\n",
    "train_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 对Age进行区间划分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>S</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Survived  Pclass  Sex  Age  SibSp  Parch     Fare Embarked  Title\n",
       "0         0       3    0    1      1      0   7.2500        S      1\n",
       "1         1       1    1    2      1      0  71.2833        C      3\n",
       "2         1       3    1    1      0      0   7.9250        S      2\n",
       "3         1       1    1    2      1      0  53.1000        S      3\n",
       "4         0       3    0    2      0      0   8.0500        S      1"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df['AgeBand'] = pd.cut(train_df['Age'], 5)\n",
    "train_df[['AgeBand', 'Survived']].groupby(['AgeBand'], as_index=False).mean().sort_values(by='AgeBand', ascending=True)\n",
    "\n",
    "# 区间数据转化\n",
    "for dataset in combine:    \n",
    "    dataset.loc[ dataset['Age'] <= 16, 'Age'] = 0\n",
    "    dataset.loc[(dataset['Age'] > 16) & (dataset['Age'] <= 32), 'Age'] = 1\n",
    "    dataset.loc[(dataset['Age'] > 32) & (dataset['Age'] <= 48), 'Age'] = 2\n",
    "    dataset.loc[(dataset['Age'] > 48) & (dataset['Age'] <= 64), 'Age'] = 3\n",
    "    dataset.loc[ dataset['Age'] > 64, 'Age']\n",
    "\n",
    "# 移除AgeBand特征\n",
    "train_df = train_df.drop(['AgeBand'], axis=1)\n",
    "combine = [train_df, test_df]\n",
    "\n",
    "train_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 通过现有特征构造新特征\n",
    "- 通过子女和配偶数量（Parch and SibSp）构造家庭成员数量特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>FamilySize</th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0.724138</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>0.578431</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>0.552795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>0.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.303538</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0.200000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>0.136364</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>11</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   FamilySize  Survived\n",
       "3           4  0.724138\n",
       "2           3  0.578431\n",
       "1           2  0.552795\n",
       "6           7  0.333333\n",
       "0           1  0.303538\n",
       "4           5  0.200000\n",
       "5           6  0.136364\n",
       "7           8  0.000000\n",
       "8          11  0.000000"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for dataset in combine:\n",
    "    dataset['FamilySize'] = dataset['SibSp'] + dataset['Parch'] + 1\n",
    "\n",
    "train_df[['FamilySize', 'Survived']].groupby(['FamilySize'], as_index=False).mean().sort_values(by='Survived', ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 构造另外一个特征IsAlone"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>IsAlone</th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0.505650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.303538</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   IsAlone  Survived\n",
       "0        0  0.505650\n",
       "1        1  0.303538"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for dataset in combine:\n",
    "    dataset['IsAlone'] = 0\n",
    "    dataset.loc[dataset['FamilySize'] == 1, 'IsAlone'] = 1\n",
    "\n",
    "train_df[['IsAlone', 'Survived']].groupby(['IsAlone'], as_index=False).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 移除旧特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Title</th>\n",
       "      <th>IsAlone</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>S</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>S</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Survived  Pclass  Sex  Age     Fare Embarked  Title  IsAlone\n",
       "0         0       3    0    1   7.2500        S      1        0\n",
       "1         1       1    1    2  71.2833        C      3        0\n",
       "2         1       3    1    1   7.9250        S      2        1\n",
       "3         1       1    1    2  53.1000        S      3        0\n",
       "4         0       3    0    2   8.0500        S      1        1"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df = train_df.drop(['Parch', 'SibSp', 'FamilySize'], axis=1)\n",
    "test_df = test_df.drop(['Parch', 'SibSp', 'FamilySize'], axis=1)\n",
    "combine = [train_df, test_df]\n",
    "\n",
    "train_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 构造新特征Age*Class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age*Class</th>\n",
       "      <th>Age</th>\n",
       "      <th>Pclass</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Age*Class  Age  Pclass\n",
       "0          3    1       3\n",
       "1          2    2       1\n",
       "2          3    1       3\n",
       "3          2    2       1\n",
       "4          6    2       3\n",
       "5          3    1       3\n",
       "6          3    3       1\n",
       "7          0    0       3\n",
       "8          3    1       3\n",
       "9          0    0       2"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for dataset in combine:\n",
    "    dataset['Age*Class'] = dataset.Age * dataset.Pclass\n",
    "\n",
    "train_df.loc[:, ['Age*Class', 'Age', 'Pclass']].head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 补全类别特征\n",
    "Embarked有三个值，用出现最频繁的S进行补全"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Survived</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>C</td>\n",
       "      <td>0.553571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Q</td>\n",
       "      <td>0.389610</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>S</td>\n",
       "      <td>0.339009</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Embarked  Survived\n",
       "0        C  0.553571\n",
       "1        Q  0.389610\n",
       "2        S  0.339009"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "freq_port = train_df.Embarked.dropna().mode()[0]\n",
    "freq_port\n",
    "\n",
    "for dataset in combine:\n",
    "    dataset['Embarked'] = dataset['Embarked'].fillna(freq_port)\n",
    "    \n",
    "train_df[['Embarked', 'Survived']].groupby(['Embarked'], as_index=False).mean().sort_values(by='Survived', ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 把类别特征转化为数值型\n",
    "Embarked特征进行简单映射"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Title</th>\n",
       "      <th>IsAlone</th>\n",
       "      <th>Age*Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Survived  Pclass  Sex  Age     Fare  Embarked  Title  IsAlone  Age*Class\n",
       "0         0       3    0    1   7.2500         0      1        0          3\n",
       "1         1       1    1    2  71.2833         1      3        0          2\n",
       "2         1       3    1    1   7.9250         0      2        1          3\n",
       "3         1       1    1    2  53.1000         0      3        0          2\n",
       "4         0       3    0    2   8.0500         0      1        1          6"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for dataset in combine:\n",
    "    dataset['Embarked'] = dataset['Embarked'].map( {'S': 0, 'C': 1, 'Q': 2} ).astype(int)\n",
    "\n",
    "train_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 补全和类型转换\n",
    "- 用中位数补全测试集中的Fare特征\n",
    "- 对Fare进行区间划分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "test_df['Fare'].fillna(test_df['Fare'].dropna().median(), inplace=True)\n",
    "test_df.head()\n",
    "\n",
    "train_df['FareBand'] = pd.qcut(train_df['Fare'], 4)\n",
    "train_df[['FareBand', 'Survived']].groupby(['FareBand'], as_index=False).mean().sort_values(by='FareBand', ascending=True)\n",
    "\n",
    "for dataset in combine:\n",
    "    dataset.loc[ dataset['Fare'] <= 7.91, 'Fare'] = 0\n",
    "    dataset.loc[(dataset['Fare'] > 7.91) & (dataset['Fare'] <= 14.454), 'Fare'] = 1\n",
    "    dataset.loc[(dataset['Fare'] > 14.454) & (dataset['Fare'] <= 31), 'Fare']   = 2\n",
    "    dataset.loc[ dataset['Fare'] > 31, 'Fare'] = 3\n",
    "    dataset['Fare'] = dataset['Fare'].astype(int)\n",
    "\n",
    "train_df = train_df.drop(['FareBand'], axis=1)\n",
    "combine = [train_df, test_df]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据处理的结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Title</th>\n",
       "      <th>IsAlone</th>\n",
       "      <th>Age*Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Survived  Pclass  Sex  Age  Fare  Embarked  Title  IsAlone  Age*Class\n",
       "0         0       3    0    1     0         0      1        0          3\n",
       "1         1       1    1    2     3         1      3        0          2\n",
       "2         1       3    1    1     1         0      2        1          3\n",
       "3         1       1    1    2     3         0      3        0          2\n",
       "4         0       3    0    2     1         0      1        1          6\n",
       "5         0       3    0    1     1         2      1        1          3\n",
       "6         0       1    0    3     3         0      1        1          3\n",
       "7         0       3    0    0     2         0      4        0          0\n",
       "8         1       3    1    1     1         0      3        0          3\n",
       "9         1       2    1    0     2         1      3        0          0"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Title</th>\n",
       "      <th>IsAlone</th>\n",
       "      <th>Age*Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>892</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>893</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>894</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>895</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>896</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>897</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>898</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>899</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>900</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>901</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass  Sex  Age  Fare  Embarked  Title  IsAlone  Age*Class\n",
       "0          892       3    0    2     0         2      1        1          6\n",
       "1          893       3    1    2     0         0      3        0          6\n",
       "2          894       2    0    3     1         2      1        1          6\n",
       "3          895       3    0    1     1         0      1        1          3\n",
       "4          896       3    1    1     1         0      3        0          3\n",
       "5          897       3    0    0     1         0      1        1          0\n",
       "6          898       3    1    1     0         2      2        1          3\n",
       "7          899       2    0    1     2         0      1        0          2\n",
       "8          900       3    1    1     0         1      3        1          3\n",
       "9          901       3    0    1     2         0      1        0          3"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 模型训练预测解决问题\n",
    "\n",
    "现在我们用特定的方法来训练预测。有60多种模型算法可供选择。需要根据问题所需来挑选合适的模型。我们的问题是分类回归问题，需要识别输出（是否存活）和其他特征（性别、年龄、登船口等）的关系。可以用有监督的机器学习方法。属于两个范畴 - 有监督学习和分类回归。列出了下面一些方法：\n",
    "\n",
    "1. Logistic Regression\n",
    "2. KNN or k-Nearest Neighbors\n",
    "3. Support Vector Machines\n",
    "4. Naive Bayes classifier\n",
    "5. Decision Tree\n",
    "6. Random Forrest\n",
    "7. Perceptron\n",
    "8. Artificial neural network\n",
    "9. RVM or Relevance Vector Machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((891, 8), (891,), (418, 8))"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train = train_df.drop(\"Survived\", axis=1)\n",
    "Y_train = train_df[\"Survived\"]\n",
    "X_test  = test_df.drop(\"PassengerId\", axis=1).copy()\n",
    "X_train.shape, Y_train.shape, X_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logistic Regression\n",
    "逻辑回归是在工作流早期运行的有用模型。 逻辑回归通过使用逻辑函数估计概率来衡量分类因变量（特征）与一个或多个独立变量（特征）之间的关系，其是累积逻辑分布\n",
    "\n",
    "- 训练集和评价集是同一个"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "80.36"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Logistic Regression\n",
    "\n",
    "logreg = LogisticRegression()\n",
    "logreg.fit(X_train, Y_train)\n",
    "Y_pred = logreg.predict(X_test)\n",
    "acc_log = round(logreg.score(X_train, Y_train) * 100, 2)\n",
    "acc_log"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用Logistic Regression来验证特征构造和补全的假设和据测，通过计算决策函数的特征相关系数来验证。\n",
    "\n",
    "正系数提高了响应的对数几率（从而增加了概率），负系数降低了响应的对数几率（从而降低了概率）。\n",
    "\n",
    "1. Sex的正系数最高，意味着Sex的值从0到1，Survived增加到1的概率最多\n",
    "2. Pclass增加，Survived存活的概率减少的越多\n",
    "3. Age*Class特征是负相关第二大的特征\n",
    "4. Title是正相关第二大的特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Feature</th>\n",
       "      <th>Correlation</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Sex</td>\n",
       "      <td>2.201527</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Title</td>\n",
       "      <td>0.398234</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Age</td>\n",
       "      <td>0.287163</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Embarked</td>\n",
       "      <td>0.261762</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>IsAlone</td>\n",
       "      <td>0.129140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Fare</td>\n",
       "      <td>-0.085150</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Age*Class</td>\n",
       "      <td>-0.311200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Pclass</td>\n",
       "      <td>-0.749007</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Feature  Correlation\n",
       "1        Sex     2.201527\n",
       "5      Title     0.398234\n",
       "2        Age     0.287163\n",
       "4   Embarked     0.261762\n",
       "6    IsAlone     0.129140\n",
       "3       Fare    -0.085150\n",
       "7  Age*Class    -0.311200\n",
       "0     Pclass    -0.749007"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coeff_df = pd.DataFrame(train_df.columns.delete(0))\n",
    "coeff_df.columns = ['Feature']\n",
    "coeff_df[\"Correlation\"] = pd.Series(logreg.coef_[0])\n",
    "\n",
    "coeff_df.sort_values(by='Correlation', ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Support Vector Machines\n",
    "\n",
    "一个二分类分类器，具有相关学习算法的监督学习模型，用于分析用于分类和回归分析，从分数上看比逻辑回归效果好"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "83.84"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Support Vector Machines\n",
    "\n",
    "svc = SVC()\n",
    "svc.fit(X_train, Y_train)\n",
    "Y_pred = svc.predict(X_test)\n",
    "acc_svc = round(svc.score(X_train, Y_train) * 100, 2)\n",
    "acc_svc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## KNN\n",
    "用于分类和回归的非参数方法。样本的预测结果根据与它相近的邻居点投票决定，样本被分配给其k个最近邻居中最常见的类别（k是正整数，通常较小）。 如果k = 1，则将对象简单地分配给该单个最近邻的类。效果比逻辑回归好，比SVM差"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "84.74"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "knn = KNeighborsClassifier(n_neighbors = 3)\n",
    "knn.fit(X_train, Y_train)\n",
    "Y_pred = knn.predict(X_test)\n",
    "acc_knn = round(knn.score(X_train, Y_train) * 100, 2)\n",
    "acc_knn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Naive Bayes classifier\n",
    "机器学习中一个简单的概率分类器，基于贝叶斯定理和特征之间的独立性假设。具有很高的可扩展性，需要一系列参数在学习问题中的变量数量（特征）的线性关系。目前为止分数最低的一个。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "72.28"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Gaussian Naive Bayes\n",
    "\n",
    "gaussian = GaussianNB()\n",
    "gaussian.fit(X_train, Y_train)\n",
    "Y_pred = gaussian.predict(X_test)\n",
    "acc_gaussian = round(gaussian.score(X_train, Y_train) * 100, 2)\n",
    "acc_gaussian"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## perceptron \n",
    "用于二进制分类器的监督学习的算法（可以决定由数字向量表示的输入是否属于某个特定类别的函数）。 它是一种线性分类器，即基于将权重集合与特征向量组合的线性预测函数进行预测的分类算法。 该算法允许在线学习，因为它一次处理训练集中的元素。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/lib64/python2.7/site-packages/sklearn/linear_model/stochastic_gradient.py:84: FutureWarning: max_iter and tol parameters have been added in <class 'sklearn.linear_model.perceptron.Perceptron'> in 0.19. If both are left unset, they default to max_iter=5 and tol=None. If tol is not None, max_iter defaults to max_iter=1000. From 0.21, default max_iter will be 1000, and default tol will be 1e-3.\n",
      "  \"and default tol will be 1e-3.\" % type(self), FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "78.0"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Perceptron\n",
    "\n",
    "perceptron = Perceptron()\n",
    "perceptron.fit(X_train, Y_train)\n",
    "Y_pred = perceptron.predict(X_test)\n",
    "acc_perceptron = round(perceptron.score(X_train, Y_train) * 100, 2)\n",
    "acc_perceptron"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "79.12"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Linear SVC\n",
    "\n",
    "linear_svc = LinearSVC()\n",
    "linear_svc.fit(X_train, Y_train)\n",
    "Y_pred = linear_svc.predict(X_test)\n",
    "acc_linear_svc = round(linear_svc.score(X_train, Y_train) * 100, 2)\n",
    "acc_linear_svc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/lib64/python2.7/site-packages/sklearn/linear_model/stochastic_gradient.py:84: FutureWarning: max_iter and tol parameters have been added in <class 'sklearn.linear_model.stochastic_gradient.SGDClassifier'> in 0.19. If both are left unset, they default to max_iter=5 and tol=None. If tol is not None, max_iter defaults to max_iter=1000. From 0.21, default max_iter will be 1000, and default tol will be 1e-3.\n",
      "  \"and default tol will be 1e-3.\" % type(self), FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "74.64"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Stochastic Gradient Descent\n",
    "\n",
    "sgd = SGDClassifier()\n",
    "sgd.fit(X_train, Y_train)\n",
    "Y_pred = sgd.predict(X_test)\n",
    "acc_sgd = round(sgd.score(X_train, Y_train) * 100, 2)\n",
    "acc_sgd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decision Tree\n",
    "使用决策树作为将特征（树枝）映射到关于目标值（树叶）的结论的预测模型。 目标变量可以采用一组有限值的树模型称为分类树; 在这些树结构中，叶表示类标签，分支表示导致这些类标签的特征的连接。 目标变量可以采用连续值（通常为实数）的决策树称为回归树。目前为止分数最高的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "86.76"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Decision Tree\n",
    "\n",
    "decision_tree = DecisionTreeClassifier()\n",
    "decision_tree.fit(X_train, Y_train)\n",
    "Y_pred = decision_tree.predict(X_test)\n",
    "acc_decision_tree = round(decision_tree.score(X_train, Y_train) * 100, 2)\n",
    "acc_decision_tree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Random Forest\n",
    "最受欢迎的模型之一。随机森林或随机决策树是一种用于分类，回归和其他任务的综合学习方法，通过在训练时间内构建多个决策树（n_estimators = 100）并输出作为类（分类）模式的类， 或平均预测（回归）。目前为止最好的模型，用于输出。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "86.76"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Random Forest\n",
    "\n",
    "random_forest = RandomForestClassifier(n_estimators=100)\n",
    "random_forest.fit(X_train, Y_train)\n",
    "Y_pred = random_forest.predict(X_test)\n",
    "random_forest.score(X_train, Y_train)\n",
    "acc_random_forest = round(random_forest.score(X_train, Y_train) * 100, 2)\n",
    "acc_random_forest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型结果汇总"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model</th>\n",
       "      <th>Score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Random Forest</td>\n",
       "      <td>86.76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Decision Tree</td>\n",
       "      <td>86.76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>KNN</td>\n",
       "      <td>84.74</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Support Vector Machines</td>\n",
       "      <td>83.84</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Logistic Regression</td>\n",
       "      <td>80.36</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Linear SVC</td>\n",
       "      <td>79.12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Perceptron</td>\n",
       "      <td>78.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Stochastic Gradient Decent</td>\n",
       "      <td>74.64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Naive Bayes</td>\n",
       "      <td>72.28</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        Model  Score\n",
       "3               Random Forest  86.76\n",
       "8               Decision Tree  86.76\n",
       "1                         KNN  84.74\n",
       "0     Support Vector Machines  83.84\n",
       "2         Logistic Regression  80.36\n",
       "7                  Linear SVC  79.12\n",
       "5                  Perceptron  78.00\n",
       "6  Stochastic Gradient Decent  74.64\n",
       "4                 Naive Bayes  72.28"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "models = pd.DataFrame({\n",
    "    'Model': ['Support Vector Machines', 'KNN', 'Logistic Regression', \n",
    "              'Random Forest', 'Naive Bayes', 'Perceptron', \n",
    "              'Stochastic Gradient Decent', 'Linear SVC', \n",
    "              'Decision Tree'],\n",
    "    'Score': [acc_svc, acc_knn, acc_log, \n",
    "              acc_random_forest, acc_gaussian, acc_perceptron, \n",
    "              acc_sgd, acc_linear_svc, acc_decision_tree]})\n",
    "models.sort_values(by='Score', ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 结果保存"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "submission = pd.DataFrame({\n",
    "        \"PassengerId\": test_df[\"PassengerId\"],\n",
    "        \"Survived\": Y_pred\n",
    "    })\n",
    "# submission.to_csv('../output/submission.csv', index=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
